It looks like the homepage “Disk Activity” display isn’t plotting any data for SAS disks. I have two HGST SAS drives connected to a SAS 9211-8i along with 6 x SATA drives. Even though data is being written to the drives, the plot is never updated (see screenshot). Currently running Rockstor 3.9.2-41
The stats not actually surfacing in /proc/diskstats (as is the case with md devices) as that is where we grab them from.
We are not usefully resolving the correct by-id name for the device, ie the /proc/diskstats uses the temp canonical type names but internally Rockstor uses the boot safe by-id type names: but there can be multiple by-id type names and for speed we take a short cut for these stats that may be coming back to bite us in this case (if I remember correctly).
The following code is the nub/core of the retrieval and includes the potential by-id reference:
with the byid_disk_map referencing the output of:
To help discern which is the cause you could give us the output of:
cat /proc/diskstats
while there is confirmed activity on the problematic reporting devices (two consecutive readings would be nice as well) and also a screen shot of your disks page to see what by-id names Rockstor is using, so we can check for (2) above.
It will also be necessary to report the full output of the following command:
ls -la /dev/disk/by-id
Note that at least all of the above requested information will be required to pin this down. But if these drives are like the md case they just don’t report their activity in that way. But I’m guessing we have a (2) type cause on this one. Just a quick guess for the time being though until get more info.
Hope that helps and thanks again for the report.
I’m assuming your last report ended up resolving itself successfully in time as there was no further update after my subsequent reply in that thread?
Best to reply contextually there I would say on that one.
It’s also odd that they don’t have the normal more multi part type name such as all the others have (bar the system disk).
Could it be that they are mapped differently (ie in the hypervisor) or are they the only 2 connected in a certain way. As I see the others with the more normal type by-id name ie ‘ata-WDC_model-serial-etc’ are also showing as SAS.
Given this looks to be a WMware install of sorts, are you sure the drives are passed through in the same was as those that are showing stats correctly.
Whatever it looks like it could still be some kind of internal Rockstor naming anomaly with regard to the stats as the given name, and that used by Rockstor, ‘wwn-0x5000cca252017870’ still resolves to /dev/sdd which in turn is still showing stats.
I’ll circle back around to this one as it’s non critical, unless some one beats me to it of course, and thanks for any more info you can give as what is different about the drives that don’t show activity in comparison to the ones that do show activity in the dashboard widget.
Also I meant to say before that we do have a pending pull request by @Flox in the rockon-registry:
for the very impressive netdata which may serve your needs for the time being. But given it’s currently pending merge you could follow the following wiki guide to try it out:
It’s a little ‘contained’ due to docker’s container type nature but seems to work fully for drive activity.
Yes, the two drives with the ID beginning with wwn are the only drives not showing in the dashboard. All drives are connected to a SAS 9211-8i card which is passed through to the VM directly. The hypervisor has no idea that there are drives connected to the PCI device and all 8 drives are connected the same way.
Pretty sure we have a (2) scenario here but we wouldn’t if those 2 drives had their expected ‘long’ by-id names. So I was trying to fathom just why that might be. Have they been added ‘live’ ie has the vm running Rockstor been rebooted since this was the case. Bit of a long shot that one. I’m just perplexed as to why they don’t have the expected names such as the others the by-id’s are dished out by udev but I don’t see why these 2 disks should be assigned different by-id names that the others.
No worries though, would just be nice to fathom why just those 2 drives are not being assigned the usual multi-part by-id.
@juchong Thanks for the head up on that new connector and power disable pin, I didn’t know about that so cheers.
OK, managed to peek in on this issue again and have something for you to try if you are game.
Obvious caveat is don’t do this if you are not confident about returning the python source file to it’s original form (ie you take a copy first or are happy enough with local editing a python source file: just have to watch spacing really)
In a recent enhancement to our main ‘proper’ by-id name selection we made an alteration that was not reflected in the quick and lightweight by-id name map created currently for use only on the dashboard.
Anyway I think if you add an ‘r’ character to the line indicated here:
ie we are simply asking for a reverse lexicographical list ordering for our by-id names.
We usually take the longest, ie the one made up from device bus + model + serial etc but given that one is for some reason missing with these new drives (although I suspect it will appear once udev is updated) then we can hopefully at least address this bug where I suspect the quick and dirty doesn’t line up with out proper by-id name selection when the only options available are the exact same length, as in your case:
Your systems local copy of that file should be at:
/opt/rockstor/src/rockstor/system/osi.py
so something like:
nano -c /opt/rockstor/src/rockstor/system/osi.py
should give you an easy editor and show the line number.
Let me know if you are game to try this change out, and for it to take effect you will need to restart your rockstor service thereafter:
systemctl restart rockstor
or reboot, and then hopefully your ‘rouge’ short name only devices activity should show up as intended.
If this works then I’ll prepare an explanation by way of issue and put this change in a pull request against that issue as their is good reason to apply it anyway.
No worries if this is all a bit messy or inappropriate for you to try on that setup, but it would help as a proof of fix to justify the time getting it documented and put in ahead of something else.
Thanks and let me know if you are game to try that change and the results either way. But do bear in mind that the change must be applied carefully and a back out plan in readiness if you are to try it. Should be pretty safe as that procedure is only used by the activity widget but of course if you break the file all bets are off. But don’t copy the file in the same place by another name as that might upset something; but you could pop it in /root or something by way of backup if needed.
As of Rockstor stable channel release version 3.9.2-42 your reported Dashboard disk activity anomaly re 2 of your disks should now be sorted via issue:
and it’s associated pull request which passed review and has now been merged:
It’s essentially that same single character change but I developed a couple of unit tests (based on your supplied command output) to go along with the change that proved the (before and after) behaviour of the associated dev name mapping function used only by the dashboard and from that it looks like you were also getting no activity indication for your system drive as well. In short if a dev had multiple by-id names (usual) that were also all the same length then we had a recent regression that lead to your reported no disk activity in the dashboard widget for those devices.
Thanks again for your report on this and do let us know if you end up working out why just those 2 devices, and as it goes your system disk, had no associated longer by-id type name as your other similarly attached devices had.
Hope that helps and let us know if I’ve missed the mark on this particular fix.
Hello. I’m a total newbie to Rockstor. I’m currently testing the AARCH64 distribution via KVM running on a 64-bit Arm system (macchiatobin). I only have a single 1 TB volume attached to Rockstor. It’s been working quite well so far with the exception of the Disk Activity plot on the dashboard which is blank.
@gsamu Welcome to the Rockstor community, and I’m glad you are enjoying the testing; much appreciated.
Your issue looks very much like there are no serial numbers assigned to your virtual disk devices. Notice that the sda (CDROM) has a by-id counterpart, where as the vda* devices do not. This is due to your virtual machine env auto assigning device serials to the emulated ATA devices, but not the libvirt devices (vda*). That is a common behaviour with VMM/libvirt qemu.
And no serial on devices pretty much writes them off for Rockstor use. See:
Try adding unique serials to all the libvirt devices with the virtual machine powered down. Then on power up the fake-serial issue should go away and you should get a far better impression of device management.
The ‘fake-serial-uuid’ reports are where we are stop-gapping with a temporary serial when we don’t find one.
Do let us know here or in a dedicated forum post if this sorts your issue as it may well b e we have another issue going on here. But first make sure you virtual env is providing unique serials to all devices and then ensure the udev is assigning by-id names to all of those devices. There after it’s down to us to use those by-id names and there may still be an issue lurking in that layer that it would be great to resolve with your help/feedback.
Hope that helps and do let us know how you get on. And thanks for the AArch64 testing. Some more details of your VM setup would be good here to enable/inform others here of how folks can test this rather new platform for us.
Thank you for the detailed response. I can confirm that after adding a serial number to the KVM virt disk, that the issue is resolved. I’m waiting on 2 x 4TB SATA drives which I’ve ordered now to have something with greater performance, capacity. I’ll write something more detailed about my experience and share here to the forum hopefully by the end of the week.
Well, it’s super high level but I threw this together at the weekend. Just a quick run down on building a NAS with Rockstor and a MACCHIATObin Arm board: https://www.gaborsamu.com/blog/aarch64_nas/
@gsamu Brilliant write-up! Interesting stuff there, I always enjoy an article like this where I am clicking in to links or scooting off into Google investigating the stuff being written about.
Nice one