Alright, a simple explanation for follow up readers of this issue:
I installed Rockstor (including the openSUSE OS) onto an USB stick - /dev/sdc
in my case.
Althought installing an OS onto an USB stick is not ideal for reliability - as seen in my case - it is not fundamentally wrong.
Plain openSUSE OS uses BTRFS as filesystem for the OS - completely independent from the btrfs filesystem used by Rockstor for the “HDD storage”.
Among other advantages, BTRFS uses a checksum for every file in order to detect hardware errors - when using other filesystem (e.g. ext4, NTFS, …) a hardware error might lead to a changed file undetected.
Every time a file gets read from a BTRFS filesystem the checksum is checked. In my case whatever process tried to access the damaged file could not start and through whatever chain of dependencies lead to the service rockstor
not even trying to start.
BTRFS in this case has done it’s job of not letting start service with corrupted files - to whatever this would have lead …
In case a RAID1-BTRFS would have been used (like for the “HDD storage”), BTRFS would of course simply load the file from the other device and and present it to the process “normally” and “just” logg this issue to the system.
The dmesg
log in my case looks like this: (sorry, I am only allowed to put one image into a post )
The “counter” of BTRFS errors can be checked on a mounted filesystem: (.
is the mounted path, sudo privilege required)
# btrfs device stats .
Append the -z
flag to reset the counter after printing the values.
I then started a scrub, where btrfs reads every single file from the device to validate the checksum:
# btrfs scrub start .
# watch btrfs srub status .
(the second command is for “watching” the progress of scrub)
After the scrub is finished I again checked (previousely reset) btrfs status:
Verdict
My (new) USB stick is damaged and this caused the issue.
… although I bought it from a “well-known” brand (Intenso)
… although it is the “Premium” line of this brand
… and although it has a metal enclousure for “optimal heat dissapation”
(these are often-recommended things for buying a USB stick where the OS is installed onto)
Never mind, this USB stick will go back to Amazon, but it is still annoying for me having all these troubles.
I have to say, this is just one more example of almost a dozen failed USB-Sticks and micro SD-cards - I have experienced a terrible failure rate with these device (subjectively ~ 30%) - some of them immediately (like in this case), and some after some usage.
And in contrast - although heavily used - I haven’t had a single failure in any 2.5" or M.2 SSD.
These device seem to much more reliable.
Cheers, Simon