@kcomer Cheers, but in that screen shot the Disk activity drop down is set for sda3, have I missed the point again? My understanding of that widget is that you have to select the disk you want to monitor and I’m assuming you are wanting to monitor a different drive. Please put me right if I have misunderstood your point.
Cheers.
Ok so this may be a mis communication on my part, the widget shows the Top Disk and then below it says “no disk activity”, is what I was referring to the graphs below seem to work, but the area where it states no activity every some many seconds it flashes a brown bar to %100 and below it states (sdc3) changing the drop down makes no difference on my system.
@kcomer, OK I get it now, thanks. So does it fail to pickup top disk activity on just reads or writes as well. I’m afraid I’m not familiar with the workings of this part of Rockstor just yet though. Thanks for reporting. I’ve opened another issue on this so it doesn’t get forgotten. As you say it could very well be the devices names not registering on this widget. Do your twl0 names show up in the drop down as well as the system drive?
Thanks.
1 Like
So it appears to be only gather the writes and no reads for sd3 only which is on the SSD none of the devices on the raid controller show up in the drop down and neither do the twl0 show up in the disk page either in case I didn’t mention that either.
If I change to the other disk the graphs never display any data.
@kcomer Thanks for this, it is bound to help with pinning down what’s not working as it should. I have added this info to the issue so in time this should get the attention it obviously needs. I’m afraid I can’t myself look to this for a bit but at least it’s there ready for people to have a go at it if they fancy and I’m rather hoping it’s just as you proposed a dev naming issue of sorts. I don’t think that part of the code has seen much activity for quite some time so it’s only good if it gets a little attention in the not too distant future. I quite like the dash facility and it seems like quite a popular feature, but yes, a little attention is needed.
Cheers.
@kcomer Just to let you know that as of testing channel update version 3.8-10.12 the issue I was hoping could be extended to include your requirement of a custom smartctl option, ie “-d 3ware,0” ended up not including this capability. Sorry about that but it just didn’t end up being practical at the time. However so that we don’t loose track of this requirement I’ve opened another issue specifically targeted at this need: “add custom SMART parameter option”. But on the more immediate side I do think the “no SMART info on MSA70” should have improved the parsing of the SMART data as you have presented it here, once SMART is enabled that is. So that’s at least a step in the right direction.
Also note that the previously linked forum thread on S.M.A.R.T. for USB drives and specifically this post details how you could jury rig the custom option you require. But take care as this will be applied universally across all your drives and controllers. But if it’s just a matter of sensing if SMART is enabled you might try it just with “toggle_smart” function and the “available” function in the /opt/rockstor/src/rockstor/system/smart.py file. That might serve as a stop gap until this new issue receives some attention. Note that I am unsure of the affect this additional parameter may have on the non 3ware controller. That is best researched with the smartmontools project.
Please do report back on any further findings, it all helps in the long run. Thanks.
1 Like
Phil,
Thanks for the update I will report back when I have a chance to update my system. may not be able to for a day or two
Kevin
Dumb question source forge does not seem to have the update is this in testing or am I to pull this from git if git then I also am not seeing 3.8.10.12 where can I pull this release?
@kcomer Yes this is in the testing channel updates. Just make sure you have subscribed to testing channel and apply the update via the normal GUI method. Or if you are running stable then no worries as if all works out with these patches then they should end up in the next stable release. No rush though just wanted to let you know. Cheers.
1 Like
Yep I’m running the stable release, I can wait till the next release comes out. I usually only go with stable releases.
phil,
Is it safe to do a yum update now with out having to fix the python version/ I think it was python?
I updated today and and while I got something that looks a little better I am still seeing issues.
So right off the bat I get the switch to in the gui to turn on the SMART feature, I never had this before, but when I do a rescan it turns off immediately.
Next is that I am still not able to get SMART data from the disk in the GUI. I believe that this is related to rockstor not pulling twl0 as my device names. I also noticed that before I ran any smart command I had no twl devices but once I ran the first command all the devices appeared, this seems to be a feature of the smartctl command "found in a piece of documentation where it states that this is the behaviour
Anyway when going to the smart info pages in the gui and do a refresh I get this error below.
The error message: Error running a command. cmd = [’/usr/sbin/smartctl’, ‘-l’, ‘selftest’, ‘-l’, ‘selective’, u’/dev/sdd’]. rc = 4. stdout = [‘smartctl 6.2 2013-07-26 r3841 [x86_64-linux-4.3.3-1.el7.elrepo.x86_64] (local build)’, ‘Copyright © 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org’, ‘’, ‘=== START OF READ SMART DATA SECTION ===’, ‘Device does not support Self Test logging’, ‘’]. stderr = [’’]
So in taking that command displayed and doing a little tweaking I get it to put some out put not sure if it will be helpful
/usr/sbin/smartctl -d 3ware,0 -l selftest -l selective /dev/twl0
smartctl 6.2 2013-07-26 r3841 [x86_64-linux-4.3.3-1.el7.elrepo.x86_64] (local build)
Copyright © 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF READ SMART DATA SECTION ===
SMART Self-test log structure revision number 1
No self-tests have been logged. [To run self-tests, use: smartctl -t]
SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.
Attempted to update my smart.py script and no SMART working after changes if I run the script i get this error for both the modified script and the original
python smart.py
Traceback (most recent call last):
File “smart.py”, line 20, in
from osi import run_command
File “/opt/rockstor/src/rockstor/system/osi.py”, line 27, in
from exceptions import CommandException
ImportError: cannot import name CommandException
These are the lines I changed
o, e, rc = run_command([SMART, '-d', '3ware','0', '-H', '--info', '/dev/%s' % device],
o, e, rc = run_command([SMART, '-d', '3ware','0', '-a', '/dev/%s' % device], throw=False)
o, e, rc = run_command([SMART, '-d', '3ware','0', '-c', '/dev/%s' % device])
o, e, rc = run_command([SMART, '-d', '3ware','0', '-l', 'error', '/dev/%s' % device],
return run_command([SMART, '-d', '3ware','0', '-t', test, '/dev/%s' % device])
o, e, rc = run_command([SMART, '-d', '3ware','0', '--info', ('/dev/%s' % device)])
return run_command([SMART, '-d', '3ware','0', '--smart=%s' % switch, '/dev/%s' % device])
Any hints appreciated
@kcomer I’ve found myself working on the smart stuff again and just wanted to clarify the following:
Am I right in thinking that the zero in “-d 3ware,0” relates to the controller ie 0 = first controller?
And that there are a number of /dev/twl* devices created that each relate to a drive on that controller.
As you pointed out it looks like with these controllers we have to point to the /dev/twl* device rather than the /dev/sd* devices in order to get sane smart info.
Another question is how does one relate the /dev/sd* names to the /dev/twl* names? I.e. how do we get a current map of one to the other.
And what happens when there is more than one controller?
I’m a little concerned about how to activate the required devices as you point out that they only appear after the first smart command, which give us a chicken and egg situation. Maybe we can cross that bridge (or make it) once we have the other issues sorted.
Thanks for looking into this and sharing thus far.
We haven’t yet implemented custom smart options such as will be needed for the “-d 3ware,0” but there is now provision for it in the db ready. But for another issue I was looking at smart device name pre-processing and thought that we might be able to use the same mechanism to redirect calls from the regular sd* devices to the twl* devices and hence lay some ground work for these controllers. So if we can identify how to live map the sd* names that rockstor uses to the twl* names that the smart calls require then I could put a provision in for this in the smart name pre-processor. Cheers.
Sorry to not get back to you sooner but given these controllers need not only custom options but also different device names there may not be any simple option to jury rig for the time, hence my questions re smart device name mapping.
Phill
Here is a before of my /dev directory
crw-rw---- 1 root tty 7, 129 Mar 9 19:23 vcsa1
crw-rw---- 1 root tty 7, 128 Mar 9 19:23 vcsa
crw-rw---- 1 root tty 7, 1 Mar 9 19:23 vcs1
crw-rw---- 1 root tty 7, 0 Mar 9 19:23 vcs
crw------- 1 root root 248, 0 Mar 9 19:23 usbmon0
crw-rw---- 1 root disk 21, 2 Mar 9 19:23 sg2
crw-rw---- 1 root disk 21, 1 Mar 9 19:23 sg1
crw-rw---- 1 root disk 21, 0 Mar 9 19:23 sg0
crw-rw---- 1 root disk 21, 8 Mar 9 19:23 sg8
crw-rw---- 1 root disk 21, 7 Mar 9 19:23 sg7
crw-rw---- 1 root disk 21, 6 Mar 9 19:23 sg6
crw-rw---- 1 root disk 21, 5 Mar 9 19:23 sg5
crw-rw---- 1 root disk 21, 4 Mar 9 19:23 sg4
crw-rw---- 1 root disk 21, 3 Mar 9 19:23 sg3
crw------- 1 root root 247, 1 Mar 9 19:23 hidraw1
crw------- 1 root root 247, 2 Mar 9 19:23 hidraw2
crw-rw---- 1 root lp 99, 0 Mar 9 19:23 parport0
crw------- 1 root root 10, 232 Mar 9 19:23 kvm
crw------- 1 root root 250, 0 Mar 9 19:23 watchdog0
crw------- 1 root root 10, 130 Mar 9 19:23 watchdog
drwxr-xr-x 4 root root 220 Mar 9 19:23 input
brw-rw---- 1 root disk 8, 0 Mar 9 19:23 sda
brw-rw---- 1 root disk 8, 3 Mar 9 19:23 sda3
brw-rw---- 1 root disk 8, 2 Mar 9 19:23 sda2
brw-rw---- 1 root disk 8, 1 Mar 9 19:23 sda1
crw-rw---- 1 root tty 7, 133 Mar 9 19:23 vcsa5
crw-rw---- 1 root tty 7, 132 Mar 9 19:23 vcsa4
crw-rw---- 1 root tty 7, 131 Mar 9 19:23 vcsa3
crw-rw---- 1 root tty 7, 130 Mar 9 19:23 vcsa2
crw-rw---- 1 root tty 7, 5 Mar 9 19:23 vcs5
crw-rw---- 1 root tty 7, 3 Mar 9 19:23 vcs3
crw-rw---- 1 root tty 7, 2 Mar 9 19:23 vcs2
drwxr-xr-x 20 root root 3540 Mar 9 19:23 .
crw-rw---- 1 root tty 7, 134 Mar 9 19:23 vcsa6
crw-rw---- 1 root tty 7, 6 Mar 9 19:23 vcs6
crw-rw---- 1 root tty 7, 4 Mar 9 19:23 vcs4
drwxr-xr-x 2 root root 3460 Mar 9 19:23 char
dr-xr-xr-x. 1 root root 204 Mar 9 19:25 …
crw-rw-rw- 1 root tty 5, 2 Mar 9 19:29 ptmx
Then I run this one command
smartctl -a -d 3ware,0 /dev/twl0
and I get the devices listed below
crw–w---- 1 root tty 4, 62 Mar 9 19:23 tty62
crw-rw---- 1 root tty 7, 129 Mar 9 19:23 vcsa1
crw-rw---- 1 root tty 7, 128 Mar 9 19:23 vcsa
crw-rw---- 1 root tty 7, 1 Mar 9 19:23 vcs1
crw-rw---- 1 root tty 7, 0 Mar 9 19:23 vcs
crw------- 1 root root 248, 0 Mar 9 19:23 usbmon0
crw-rw---- 1 root disk 21, 2 Mar 9 19:23 sg2
crw-rw---- 1 root disk 21, 1 Mar 9 19:23 sg1
crw-rw---- 1 root disk 21, 0 Mar 9 19:23 sg0
crw-rw---- 1 root disk 21, 8 Mar 9 19:23 sg8
crw-rw---- 1 root disk 21, 7 Mar 9 19:23 sg7
crw-rw---- 1 root disk 21, 6 Mar 9 19:23 sg6
crw-rw---- 1 root disk 21, 5 Mar 9 19:23 sg5
crw-rw---- 1 root disk 21, 4 Mar 9 19:23 sg4
crw-rw---- 1 root disk 21, 3 Mar 9 19:23 sg3
crw------- 1 root root 247, 1 Mar 9 19:23 hidraw1
crw------- 1 root root 247, 2 Mar 9 19:23 hidraw2
crw-rw---- 1 root lp 99, 0 Mar 9 19:23 parport0
crw------- 1 root root 10, 232 Mar 9 19:23 kvm
crw------- 1 root root 250, 0 Mar 9 19:23 watchdog0
crw------- 1 root root 10, 130 Mar 9 19:23 watchdog
drwxr-xr-x 4 root root 220 Mar 9 19:23 input
brw-rw---- 1 root disk 8, 0 Mar 9 19:23 sda
brw-rw---- 1 root disk 8, 3 Mar 9 19:23 sda3
brw-rw---- 1 root disk 8, 2 Mar 9 19:23 sda2
brw-rw---- 1 root disk 8, 1 Mar 9 19:23 sda1
crw-rw---- 1 root tty 7, 133 Mar 9 19:23 vcsa5
crw-rw---- 1 root tty 7, 132 Mar 9 19:23 vcsa4
crw-rw---- 1 root tty 7, 131 Mar 9 19:23 vcsa3
crw-rw---- 1 root tty 7, 130 Mar 9 19:23 vcsa2
crw-rw---- 1 root tty 7, 5 Mar 9 19:23 vcs5
crw-rw---- 1 root tty 7, 3 Mar 9 19:23 vcs3
crw-rw---- 1 root tty 7, 2 Mar 9 19:23 vcs2
crw-rw---- 1 root tty 7, 134 Mar 9 19:23 vcsa6
crw-rw---- 1 root tty 7, 6 Mar 9 19:23 vcs6
crw-rw---- 1 root tty 7, 4 Mar 9 19:23 vcs4
drwxr-xr-x 2 root root 3460 Mar 9 19:23 char
dr-xr-xr-x. 1 root root 204 Mar 9 19:25 …
crw-rw-rw- 1 root tty 5, 2 Mar 9 19:39 ptmx
crw------- 1 root root 244, 9 Mar 9 19:39 twl9
crw------- 1 root root 244, 8 Mar 9 19:39 twl8
crw------- 1 root root 244, 7 Mar 9 19:39 twl7
crw------- 1 root root 244, 6 Mar 9 19:39 twl6
crw------- 1 root root 244, 5 Mar 9 19:39 twl5
crw------- 1 root root 244, 4 Mar 9 19:39 twl4
crw------- 1 root root 244, 3 Mar 9 19:39 twl3
crw------- 1 root root 244, 2 Mar 9 19:39 twl2
crw------- 1 root root 244, 15 Mar 9 19:39 twl15
crw------- 1 root root 244, 14 Mar 9 19:39 twl14
crw------- 1 root root 244, 13 Mar 9 19:39 twl13
crw------- 1 root root 244, 12 Mar 9 19:39 twl12
crw------- 1 root root 244, 11 Mar 9 19:39 twl11
crw------- 1 root root 244, 10 Mar 9 19:39 twl10
crw------- 1 root root 244, 1 Mar 9 19:39 twl1
crw------- 1 root root 244, 0 Mar 9 19:39 twl0
drwxr-xr-x 20 root root 3860 Mar 9 19:39 .
and I get this output also if I try to run twl1, 2,3 etc I get nothing so not sure why I get an output for twl0 and nothing else.
[root@rockstor dev]# smartctl -a -d 3ware,0 /dev/twl0
smartctl 6.2 2013-07-26 r3841 [x86_64-linux-4.3.3-1.el7.elrepo.x86_64] (local build)
Copyright © 2002-13, Bruce Allen, Christian Franke, www.smartmontools.org
=== START OF INFORMATION SECTION ===
Model Family: Western Digital RE4 (SATA 6Gb/s)
Device Model: WDC WDxxxxxxxxxxxxxxxxx
Serial Number: WD-WCCxxxxxxxxx
LU WWN Device Id: 5 00xxxx xxxxxxxx
Firmware Version: 01.01K02
User Capacity: 2,000,398,934,016 bytes [2.00 TB]
Sector Size: 512 bytes logical/physical
Rotation Rate: 7200 rpm
Device is: In smartctl database [for details use: -P show]
ATA Version is: ATA8-ACS (minor revision not indicated)
SATA Version is: SATA 3.0, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is: Wed Mar 9 19:41:43 2016 EST
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED
General SMART Values:
Offline data collection status: (0x82) Offline data collection activity
was completed without error.
Auto Offline Data Collection: Enabled.
Self-test execution status: ( 0) The previous self-test routine completed
without error or no self-test has ever
been run.
Total time to complete Offline
data collection: (25200) seconds.
Offline data collection
capabilities: (0x7b) SMART execute Offline immediate.
Auto Offline data collection on/off support.
Suspend Offline collection upon new
command.
Offline surface scan supported.
Self-test supported.
Conveyance Self-test supported.
Selective Self-test supported.
SMART capabilities: (0x0003) Saves SMART data before entering
power-saving mode.
Supports SMART auto save timer.
Error logging capability: (0x01) Error logging supported.
General Purpose Logging supported.
Short self-test routine
recommended polling time: ( 2) minutes.
Extended self-test routine
recommended polling time: ( 275) minutes.
Conveyance self-test routine
recommended polling time: ( 5) minutes.
SCT capabilities: (0x70bd) SCT Status supported.
SCT Error Recovery Control supported.
SCT Feature Control supported.
SCT Data Table supported.
SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE
1 Raw_Read_Error_Rate 0x002f 200 200 051 Pre-fail Always - 0
3 Spin_Up_Time 0x0027 167 161 021 Pre-fail Always - 6641
4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 109
5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0
7 Seek_Error_Rate 0x002e 200 200 000 Old_age Always - 0
9 Power_On_Hours 0x0032 088 088 000 Old_age Always - 8891
10 Spin_Retry_Count 0x0032 100 100 000 Old_age Always - 0
11 Calibration_Retry_Count 0x0032 100 100 000 Old_age Always - 0
12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 109
183 Runtime_Bad_Block 0x0032 100 100 000 Old_age Always - 0
192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 108
193 Load_Cycle_Count 0x0032 200 200 000 Old_age Always - 1
194 Temperature_Celsius 0x0022 117 086 000 Old_age Always - 33
196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0
197 Current_Pending_Sector 0x0032 200 200 000 Old_age Always - 0
198 Offline_Uncorrectable 0x0030 200 200 000 Old_age Offline - 0
199 UDMA_CRC_Error_Count 0x0032 200 200 000 Old_age Always - 0
200 Multi_Zone_Error_Rate 0x0008 200 200 000 Old_age Offline - 0
SMART Error Log Version: 1
No Errors Logged
SMART Self-test log structure revision number 1
No self-tests have been logged. [To run self-tests, use: smartctl -t]
SMART Selective self-test log data structure revision number 1
SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS
1 0 0 Not_testing
2 0 0 Not_testing
3 0 0 Not_testing
4 0 0 Not_testing
5 0 0 Not_testing
Selective self-test flags (0x0):
After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.
Hope this helps
Ok Figured out the command
So note the highlighted number changes for each disk to get the info per disk, it has been awhile since I messed with this
smartctl -a -d 3ware,0 /dev/twl0
smartctl -a -d 3ware,1 /dev/twl0
smartctl -a -d 3ware,6 /dev/twl0
Reference to this site
https://www.schirmacher.de/display/INFO/Configuring+smartmontools+for+LSI+9750+RAID+Controller+on+XenServer
@kcomer Thanks this is exactly the info / explanation I was after. Sorry for the slow response but I wanted to report back after some progress had been made on the information you provided.
Just to let you know that the issue opened in part down to this thread add custom SMART parameter option has now been closed.
This again is only available in the testing channel for the time being. On the raid controller front it’s fairly manual but if you have a one to one mapping of your physical drives to the disks displayed in Storage - Disks then it should now be possible from 3.8-12.04 (testing channel) onwards to add drive specific raid smart options such as you have detailed in this thread and get a great deal more S.M.A.R.T info.
Just to re-itterate your findings in the context of these new drive specific smart options one would have to enter a custom 3ware port -d option for every drive ie:
First lsi 3ware controller smart target devcie = /dev/twl0 so we add this to every drive as the raid target device and just change the port to that appropriate for the drive.
drive on port 0 on first controller:- -d 3ware,0 /dev/twl0
drive on port 1 on first controller:- -d 3ware,1 /dev/twl0
drive on port 2 on first controller:- -d 3ware,2 /dev/twl0
Just as you described already.
So hence the one to one mapping requirement. Rockstor internally will apply these custom options to all the usual commands ie ‘-s on’ and ‘-a’ etc.
The change should account for the other target raid device names associated with other supported cards so hopefully it should be possible for all smartmontools supported raid cards to now get full smart info via Rockstor’s UI using the appropriate custom smart option.
The new custom options system accounts for the altered raid syntax by looking for a supplied raid target device and switches to the raid syntax if one is found, if not then it will use the device name displayed on the Disks page.
Hope this helps and I appreciate that you are on the stable branch, and thanks again for helping with these developments, so you may not be able to test this for a bit but do let the forum know if you find any problems when the time comes. I’m afraid there is no automatic discovery of these options as yet but it’s a start that we now have custom smart options capability. The configuration page links to helpful upstream documentation as to what switches do what and there is some input validation.
Picture of the custom optins entry form:
Hi Kevin,
I got the same error executing “python smart.py” as described here. Did you solve the issue? Would that be the reason why rockstor does not show any smart status for my drives at all
No S.M.A.R.T Capabilities found (ATA/SATA only)
No S.M.A.R.T Attributes found (ATA/SATA only)
even though I can see smart is activated via smartctl
` > smartctl -a /dev/sda
[…]
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
`
Ok. I found the answer here Disk attributes - missing? . One has press refresh to get the smart attributes and other information.
Phill Beautiful work I just updated my syste to 3.8.13 and the custom settings worked like a charm. Right out of the box so to speak. Added the necessary options and I get my data as expect.
Thanks So much for all the hard work
Kevin
2 Likes
@kcomer Thanks for reporting this, I’m chuffed it’s finally available and working for you. Could you post a screen shot of the disks page showing the raid settings as this may serve as a quick look at example for others to see how to configure this, no worries otherwise though. It turned out to be quite a journey and a bit more involved than I had originally anticipated.
Thanks again for your help on getting this done.
Definitely room for improvement but heading in the right direction, hopefully.
1 Like