WD drives showing as unknown

cferra · July 22, 2018, 5:32am

So i have 2 WD drives, model numbers WDC_WD80EZZX-11CSGA0 i was able to get them working and added to the pool and everything however they continually show up as greyed out and unknown in the disk list. They appear to be active and working and data seems to be read and written to them but them showing up as unknown is concerning.

I am running a LSI 9211-8i in IR - running the latest firmware and BIOS.

Running the latest stable release 3.9.2-31

I’ve tried the megaraid argument in the SMART settings but either my card doesn’t need them or the commands are not supported, because when I enter them SMART then shows as unsupported for the drive.

Without the custom options at least smart shows up and I can run quick and extended tests, when the tests are running the drives show up as active in the disk list.

phillxnet · July 22, 2018, 11:03am

@cferra Welcome to the Rockstor community, and thanks for helping to support Rockstor development via a stable subscription.

I’m assuming here you are referring to the drives “Power Status” column entry. It’s the only “unknown” Web-UI column entry on the Disks page that comes to mind, ie:

unknown-power-status

If so this is simply a default state if a certain hdparm command returns an error or a result which is not ‘understood’. The code in question is the following:

github.com

rockstor/rockstor-core/blob/master/src/rockstor/system/osi.py#L1365-L1398


      
          
          
def clean_system_rtc_wake():
              # Every time we write to rtc alarm file this get locked and
              # we have to clean it with a 0 before writing another epoch
              with open(RTC_WAKE_FILE, "w") as rtc:
                  # TODO: Test the following as: 'rtc.write("0")' may also work.
                  rtc.write("{}".format(0))
          
          

          
def set_system_rtc_wake(wakeup_epoch):
              # This new function receive desired current and wake up time
              # and set right epoch time to rtc alarm file.
              # Epoch wake up time evaluated on every shutdown scheduled task
              clean_system_rtc_wake()
              with open(RTC_WAKE_FILE, "w") as rtc:
                  rtc.write("{}".format(int(wakeup_epoch)))
              return None
          
          

          
def is_network_device_responding(address):

This file has been truncated. show original

Clicking on the “show original” will give the entire rather short code snippet that deals with the “Power State” column entry text.

This is not really a concern as it just means the output of the following command, as executed with your drives by-id name, was interpreted as ‘unknown’:

hdparm -C -q device-by-id-name-with-path

An example for a virtio drive with serial = “serial-6”:

hdparm -C -q /dev/disk/by-id/virtio-serial-6 
 drive state is:  unknown

It would also be helpful to see what the return code was from that command: ie directly after executing the hdparm command if you could then execute the following command:

echo $?
25

We see here that a return code of 25 was recorded from the hdparm command on a virtio device. The indicated source code classes any error output “len(err) !=1” as an unknown state, this error text is normally part of the command output if I remember correctly.

We may have an opportunity to improve this columns scope here (ie for your drives through LSI controller ‘unknown’ issue); or it may just be that hdparm can’t handle this scenario. We do have a related issue open for the scenario where we also use hdparm which is known not to be able to set drive power down settings (the related use of hdparm) when the drive is behind a raid controller:

https://github.com/rockstor/rockstor-core/issues/1359

Your drives by-id names can be listed via:

ls -la /dev/disk/by-id/

and are the names displayed in the Name column of the Disks page, remember to supply the path.

This is a currently not a directly related issue, ie hdparm is used for the Power Status and APM columns where as smartctl is used to establish if we can gain smart info for use in the pages that appear when you click on a drives names: as shown in our S.M.A.R.T doc section. If the smart column doesn’t show as “Not supported” and the smart pages show up ok then it would seem you don’t need custom smart options.

We have had a recent improvement on drive name validation which may affect your chosen custom smart options; see issue:
https://github.com/rockstor/rockstor-core/issues/1942
which has been initially addressed by pull request:

https://github.com/rockstor/rockstor-core/pull/1945

which was merged as part of Rockstor version 3.9.2-30 so fairly recently.

Looks like this may have impact on our parsing of LSI dev names also. The original forum member reporter of the megaraid dev parsing issue, @coreybrett, has yet to confirm the effectiveness or otherwise of these recent changes to the dev name parsing in our custom smart options re the megaraid option:

Hope that helps and lets see what your hdparm command is returning for those unknown / active drives, ie you could test the hdparm command for the drives while they are either unknown or active (if I have understood your description correctly).

cferra · August 1, 2018, 4:00pm

So I got this result - not sure if I ran it correctly –

hdparm -C -q /dev/disk/by-id/ata-WDC_80EZZX-11CSGA0_VLJHLE1Y
drive state is: unknown

echo $?
0

cferra · August 1, 2018, 4:08pm

here is a copy of what my disk UI shows.

phillxnet · August 1, 2018, 4:59pm

@cferra Thanks for the feedback and command output.

OK, so that is where the ‘unknown’ status is coming from. Essentially harmless.

It would appear that hdparm for these disks, at least how you currently have them connected (behind the LSI presumably), just can’t tell us their power state (active/idle etc). Nothing much we can do about this but in time the situation may change if this behaviour is improved upstream.

Thanks for the feedback. I wouldn’t worry about this as it’s a ‘nice to have’ Web-UI feedback element only.

Just received you Disks table picture post. At least you were able to set the power down / Spin Down setting:
“Vendor defined (8-12h)”

You could let us know if this feature works as it’s a bit hit and miss with various hardware and you may well be a candidate for the “extend custom smart parameter support” issue I linked earlier. Ie setting this option through raid controllers can be tricky.

This has reminded me that we are due for a technical wiki entry for how this element of Rockstor works. I’ll have a go at that soon but in the mean time the following was the pr that added this feature and contains a run down on how it works:

github.com/rockstor/rockstor-core

drive power down interface. Fixes #885

rockstor:master ← phillxnet:885_drive_power_down_interface

opened 03:48PM - 25 Apr 16 UTC

phillxnet

+1129 -66

Adds a per drive power down interface / feature based on idle time using the sta…ndard hdparm parameters. Given this is closely related to a drive's Advanced Power Management (APM) setting a display of the current setting and an option to change this setting is also included in the disks table and on the same configuration page. Care has been taken to graphically associate the APM setting with the Power Status (spin down) setting by grouping these columns one after the other and maintaining this relationship on the settings page also. Hdparm is itself an interface to the kernel's own drive control subsystem so we are sticking to core linux utilities for this feature. The hdparm switches used are -C to set idle spin down time and -B to read and set APM level. N.B. It is not possible to read a drives current setting for -C (idle spin down) which complicates matters, the meaning for these values can also vary between drives. The settings used are drawn from man hdparm and are apparently more reliable for newer drives. Hdparm settings are maintained over a reboot but not over a power cycle. To address this a new stand alone rockstor systemd service is introduced (rockstor-hdparm.service). It does not depend on any other rockstor systemd service and no other rockstor service depends on it. The service is used simply to execute the tested hdparm commands that are otherwise executed on demand via the WebUI. That is if no error message and no non zero return code is received from a proposed (via user config entry) hdparm command then that same command is placed in this systemd service unit to be applied on the next boot to address the power cycle loss of these settings otherwise. As idle time / spin down is associated with rotational media we do not apply any setting to a device if it fails an included is_rotational test. This test is based on a drives udevadm info readings for rpm and Automatic Acoustic Management (AAM) readings. The kernels own proc reading of rotation has been frequently reported as currently unreliable hence establishing our own mechanism for this. This function (in system/osi) is considered as conservative and will err on the "not confirmed as rotational" side. If the is_rotational function returns false no setting is applied and user confirmation of this is simply to not update our disk table with any requested settings. This could be improved upon but may benefit from future enhancements in a notification system. As is the following info level message is logged:- "Skipping hdparm settings: device not confirmed as rotational" This should soon be more readily available by improvement in WebUI log view functionality. The core of this feature's function is several light weight property extensions to the disk model that provide the power status, hdparm -C setting, and current APM level to the disks page by way of the additional 2 columns. There is also included a pause function that simply executes hdparm -y to request a drive immediately enters standby mode. No full sleep mode is activated by the included mechanisms (ie hdparm -Y) as such a state requires that a drives interface be re-set and was considered overly heavy handed and potentially more problematic. The active settings change path takes the users input and tests it for sanity then attempt to run the proposed hdparm command one switch at a time; starting with the -B setting as this is more often problematic due to some drives not supporting this APM level set option. If any errors or non zero return codes are received then this setting is not included in the final systemd entry. The -C switch is then also tested in isolation and again if any errors or non zero return codes are encountered then at this stage no entry will be submitted to the systemd file. This means in some circumstances we may have only -C settings for a device but if the -B setting passed our tests then it is by default included; often simply re-affirming the default setting unless the user has specified otherwise. The core method to update the systemd service is implemented in update_hdparm_service which is utilized only by set_disk_spindown that tests the viability of the proposed hdparm switches. To facilitate the option to not include a -B setting we use the 0 value for that switch. This is a symmetrical arrangement in that if an hdparm -B reading fails or returns 'not supported' the reading mechanism conflates these responses into the 0 value (see get_disk_APM_level). This is otherwise an unused APM value (ie 1-255) and so we can maintain a simple interger for this value. This way we avoid applying any -B option to a drive that has otherwise failed to cleanly respond to our -B request or has done so but with a 'not supported' response. Like wise the mechanism is employed to remove any existing -B entry by passing zero as the requested value. The same reading mechanism translates the off reading to 255 which requires our interface logic to translate this back to 'off' but this was considered a cleaner system than dealing with the raw values of 0 (our flag) or 1-255 (written values) or 'off' which represents a reading of the the written value 255. This way we maintain symmetry with reading (using our low level reader) and writing and only translate in the UI for readability. On the UI front the APM settings are disabled by way of their tickbox option and the tick box is also disabled when an APM read of 0 (error or not supported) is encountered. Note that all hdparm commands use the dev by-id names to maintain correct targets over system reboots. Several general purpose mechanism have been included in this pr to facilitate translating the currently db drive.names of sda / sdb to these more stable by-id drive names which are created by the udev subsystem and represent simple symbolic links to their associated sda / sdb type names. It is intended that these translation functions be employed on a wider basis going forward. By default we offer a 20 minute idle spin down time if a prior setting is not found. Given prior settings cannot be established using system calls we establish the single point of truth to be direct reading from the device and the systemd file where reading from the device is not possible (ie with -C settings). This avoids using the django db and was intended to keep the whole hdparm feature as light as possible. If the systemd file does not exist it is created from the empty template held in /opt/rockstor/conf/rockstor-hdparm.service. When newly established the service is also enabled but not started as the associated commands have already been executed by way of the tests prior to their entry in the systemd file. The service being enabled is sufficient to ensure that they will be run on next boot which is all that is needed. If on the other hand no entries are found in an existing /etc/systemd/system/rockstor-hdparm.service file, ie by way of UI instigated edits, then the service is disabled and the service file removed in order to prepare for new settings and the re enabling of the service in that instance. In light of issue #1275 and it's associated pr #1287 which re-factors the disks table creation code modified in this pr I have prepared some handlebar helpers and included them in this pull request, they are however untested as they rely on what is as yet un-merged UI code. Testing of the supplied spin-down values has been mostly successful with values of 20 minutes and less as higher values are less reliable to non functional. It was also found that with no WebUI open the drives are much more reliably suspended, however opening the WebUI and viewing the disks page after drives have entered standy mode and even refreshing this view via browser or via use of the "Rescan" button doesn't then wake the drives that have entered standby. So there appears to be some command active that inhibit the drives entering standby (while the WebUI is open) but doesn't then activate these drives once they are in standby mode. This was a hard one find. Also note that even with the WebUI open multiple drives did successfully enter standby when set to 30 seconds or 1 minute but no longer. However 20 mins and below seem to work find when the WebUI isn't active. This has mostly been tested when viewing the disks page and is quite a time consuming business. Hence the submission of this pr at the beginning of a testing cycle. Reference in the user interface is also made to the recently merged "Task execution time windows" feature. Note: This pr submission text is also intended as the basis of a wiki entry upon successful review and merge. The wiki entry can then serve as a technical manual on this subsystem.

Hope that helps and I’ll try and get that wiki entry done soon.