Scanning disks not working

I had a working installation but at some point it stopped being able to mount.
When this error occured I tried a reinstall but it still happens:

            Traceback (most recent call last):
  File "/opt/rockstor/src/rockstor/rest_framework_custom/generic_view.py", line 41, in _handle_exception
    yield
  File "/opt/rockstor/src/rockstor/storageadmin/views/disk.py", line 418, in post
    return self._update_disk_state()
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/contextlib.py", line 81, in inner
    return func(*args, **kwds)
           ^^^^^^^^^^^^^^^^^^^
  File "/opt/rockstor/src/rockstor/storageadmin/views/disk.py", line 115, in _update_disk_state
    attached_disks = scan_disks(MIN_DISK_SIZE)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/rockstor/src/rockstor/system/osi.py", line 330, in scan_disks
    blk_dev_properties: dict = {
                               ^
  File "/opt/rockstor/src/rockstor/system/osi.py", line 336, in <dictcomp>
    for key, value in (
        ^^^^^^^^^^
ValueError: not enough values to unpack (expected 2, got 1)

This is the output of lsblk which osi.py seems to parse:

NAME="/dev/sda" MODEL="WDC WD40EFAX-68JH4N1" SERIAL="WD-WX42D31R3YSX" SIZE="3.6T" TRAN="sata" VENDOR="ATA     " HCTL="0:0:0:0" TYPE="disk" FSTYPE="btrfs" LABEL="data" UUID="9edc4433-e5cb-4440-a11a-1db81dd567b5"
NAME="/dev/sdb" MODEL="ST4000VN008-2DR166" SERIAL="ZGYA07DT" SIZE="3.6T" TRAN="sata" VENDOR="ATA     " HCTL="1:0:0:0" TYPE="disk" FSTYPE="btrfs" LABEL="data" UUID="9edc4433-e5cb-4440-a11a-1db81dd567b5"
NAME="/dev/sdc" MODEL="WDC WD40EFAX-68JH4N1" SERIAL="WD-WX42D31R3K6Y" SIZE="3.6T" TRAN="sata" VENDOR="ATA     " HCTL="4:0:0:0" TYPE="disk" FSTYPE="btrfs" LABEL="data" UUID="9edc4433-e5cb-4440-a11a-1db81dd567b5"
NAME="/dev/sdd" MODEL="SanDisk 3.2Gen1" SERIAL="01016c73a2fabf22051f78f30546627c9fb5e8083fbc14d477f85d6a269220dd15710000000000000000000060a25f5700804f0091558107a3ab6843" SIZE="57.3G" TRAN="usb" VENDOR=" USB    " HCTL="6:0:0:0" TYPE="disk" FSTYPE="" LABEL="" UUID=""
NAME="/dev/sdd1" MODEL="" SERIAL="" SIZE="2M" TRAN="" VENDOR="" HCTL="" TYPE="part" FSTYPE="" LABEL="" UUID=""
NAME="/dev/sdd2" MODEL="" SERIAL="" SIZE="64M" TRAN="" VENDOR="" HCTL="" TYPE="part" FSTYPE="vfat" LABEL="EFI" UUID="1381-2428"
NAME="/dev/sdd3" MODEL="" SERIAL="" SIZE="2G" TRAN="" VENDOR="" HCTL="" TYPE="part" FSTYPE="swap" LABEL="SWAP" UUID="ee6de8d5-eb1c-4d19-943c-0d24ecee92a7"
NAME="/dev/sdd4" MODEL="" SERIAL="" SIZE="55.2G" TRAN="" VENDOR="" HCTL="" TYPE="part" FSTYPE="btrfs" LABEL="ROOT" UUID="46befa9e-b399-480d-b10d-338a4c875615"
NAME="/dev/sde" MODEL="WDC WD40EFAX-68JH4N1" SERIAL="WD-WX42D31R32K3" SIZE="3.6T" TRAN="sata" VENDOR="ATA     " HCTL="5:0:0:0" TYPE="disk" FSTYPE="btrfs" LABEL="data" UUID="9edc4433-e5cb-4440-a11a-1db81dd567b5"
NAME="/dev/sdf" MODEL="DT microDuo 3C" SERIAL="1C1B0D0194C9E580E9470022" 

I’ve tried making scan_disks more robust but I just don’t seem to find what is causing this. I have noticed a couple of weird spaces in some of the output but can’t really figure it out.
On master there is a change to how this works. I’ve tried replacing my current osi.py with the most recent one but that also fails.

The btrfs array is still alive and well and I can mount it without any problems

2 Likes

I’ve done some debugging and the leading space for the vendor of /dev/sdd is what breaks it all
Debugging further

2 Likes

Hi @cellisten ,

Nice investigation so far… That reminds me of a big that was recently fixed in:

Could it be you are running a version of Rockstor prior to that fix?

1 Like

Might very well be, didn’t realise the image didn’t contain the latest version of rockstor. I had updated to latest version in the previous install.
I did some refactoring of the generation of the blk_dev_properties to separate the line in a manner where spaces don’t really matter and the strip is done differently:

line = [ item.strip() for item in re.split(r'["=]', line) ]
line = [ item for i, item in enumerate(line,-1) if i %3 != 0 ]
# Device information built from each lsblk line in turn.
blk_dev_properties: dict = {
key.lower() if key != "TRAN" else "transport":
value if value != "" else None for key, value in zip(line[::2], line[1::2])
}

But doesn’t seem changes are picked up immediately when editing osi.py. For some reason, rockstor-pre fails to restart after editing, is there a checksum check or something?

The latest version does not help, I have verified that the new line doesn’t strip leading spaces (which my code example does):

line = 'NAME="/dev/sdd" MODEL="SanDisk 3.2Gen1" SERIAL="01016c73a2fabf22051f78f30546627c9fb5e8083fbc14d477f85d6a269220dd15710000000000000000000060a25f5700804f0091558107a3ab6843" SIZE="57.3G" TRAN="usb" VENDOR=" USB    " HCTL="6:0:0:0" TYPE="disk" FSTYPE="" LABEL="" UUID=""'
>>> re.sub('"\s+"', '""', line).strip()
'NAME="/dev/sdd" MODEL="SanDisk 3.2Gen1" SERIAL="01016c73a2fabf22051f78f30546627c9fb5e8083fbc14d477f85d6a269220dd15710000000000000000000060a25f5700804f0091558107a3ab6843" SIZE="57.3G" TRAN="usb" VENDOR=" USB    " HCTL="6:0:0:0" TYPE="disk" FSTYPE="" LABEL="" UUID=""'
2 Likes

@cellisten a belated welcome to the Rockstor community. I assume, @Flox and/or @phillxnet will chime in here, but for now I have actually created an issue on the github repository referencing this thread. If it is not needed we can always close it again:

1 Like

Thanks @Hooverdan! I have now managed to build my change locally and it seems to be working. I’ll try to send a pull request for the github issue when I have time

3 Likes

That would be awesome. With the small group of regular devs, help is welcome. When and if you, take a look at the community contributions:

https://rockstor.com/docs/contribute/contribute.html#developers

There’s also a mention of a battery of tests that you can run through to ensure that at least all the standard test scenarios are not negatively impacted. Thanks!

1 Like

I’ve now sent a pull request (Refactor lsblk, resolves rockstor/rockstor-core#2907 by cellisten · Pull Request #2909 · rockstor/rockstor-core · GitHub)
The test case has been updated and verified to fail on the current code but work on the new one. I had some issues running poetry run pytest from the root of the repo (it complained about settings not being configured during collection) but I did manage to run the specific test_osi.py successfully.
I might be missing something when it comes to how to run the battery of tests.
I also added .python-version to the .gitignore to be able to run everything in a local pyenv as well.
Are the unit tests expected to be run on a running server? I’m a bit confused

2 Likes

Awesome, thanks a lot!

Just a quick answer regarding running our unit test suite:

Our test suite is using the unittest framework and not pytest so that may be the confusion here. We do have some more details in our docs on how to run the tests:
https://rockstor.com/docs/contribute/contribute.html#code-test

Hope this helps despite the brevity.

Thank you again for taking the time and effort to update the tests as well!

2 Likes

Ah, so most of the tests are system tests rather than unit tests (as they require a working installation to run). I will test on my running server as well :slight_smile:

1 Like

Ran all the tests successfully now, seems running them breaks the current config though…

2 Likes

@cellisten Thanks for checking all the test. Nice.

Re:

That is not a know issue. But note entirely impossible. If you can detail exactly what you mean here that would be good. These tests are not intended to run on a production system: they are development tests and are unit tests as I see it. Keep in mind that we require in some test a DB state of sorts: so we need to use the framework and it’s fixtures to provide known DB states. the test_osi.py and some others are simpler however. We also have some API tests etc.

So re the breakage on test runs, we would need an exact reproducer. And from a source install ideally. All good stuff here however and great to have another set of eyes on ‘stuff’. Lots to do and not may fingers between the regular contributors :slight_smile: . But we are late testing phase currently so larger, and specifically deeper changes are a worry and so may be postponed (or better approached) once we are in the next testing phase.

Hope that helps.

2 Likes

I guessed as much, knew I was taking a chance but I wanted to know what would happen. It solved itself after a restart so nothing to really worry about.
I’m used to unit tests being static tests with no dependencies, hence tending to call tests that rely on api endpoints or databases system tests. If those are known as unit tests in this project, no worries, just wanted to know what to expect. test_osi.py does behave as what I am used to unit tests being though but I haven’t been able to run it outside of the actual installed system the way it’s described in the test file:
/opt/rockstor/.venv/bin/python -m unittest test_osi.OSITests.test_scan_disks_lsblk_parse_fail
Running poetry run pytest src/rockstor/system/tests/test_osi.py works like a charm though

2 Likes