Shares are Visible in Web UI but Cannot Connect

Brief description of the problem

I have not been backing up regularly lately, and over the last couple of weeks my Rockstor appliance updated (I am on Stable updates) to 3.9.2-30. I can no longer connect to my backup shares, and am getting errors in the Web UI while navigating.

Detailed step by step instructions to reproduce the problem

Tried to use time machine to backup my MBP, and got an error saying the disk was unavailable. When navigating the Web UI to investigate, I am getting the errors below.

Web-UI screenshot

Error Traceback provided on the Web-UI

Traceback (most recent call last): File "/opt/rockstor/src/rockstor/rest_framework_custom/generic_view.py", line 41, in _handle_exception yield File "/opt/rockstor/src/rockstor/storageadmin/views/user.py", line 113, in get_queryset return combined_users() File "/opt/rockstor/src/rockstor/storageadmin/views/ug_helpers.py", line 69, in combined_users temp_uo.pincard_allowed, temp_uo.has_pincard = pincard_states(temp_uo) # noqa E501 File "/opt/rockstor/src/rockstor/system/pinmanager.py", line 147, in pincard_states pincard_present = has_pincard(user) File "/opt/rockstor/src/rockstor/system/pinmanager.py", line 130, in has_pincard pins = Pincard.objects.filter(user=int(uid_field)).count() File "/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/query.py", line 318, in count return self.query.get_count(using=self.db) File "/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/sql/query.py", line 466, in get_count number = obj.get_aggregation(using, ['__count'])['__count'] File "/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/sql/query.py", line 447, in get_aggregation result = compiler.execute_sql(SINGLE) File "/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/sql/compiler.py", line 840, in execute_sql cursor.execute(sql, params) File "/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/backends/utils.py", line 64, in execute return self.cursor.execute(sql, params) File "/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/utils.py", line 98, in __exit__ six.reraise(dj_exc_type, dj_exc_value, traceback) File "/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/backends/utils.py", line 64, in execute return self.cursor.execute(sql, params) ProgrammingError: relation "storageadmin_pincard" does not exist LINE 1: SELECT COUNT(*) AS "__count" FROM "storageadmin_pincard" WHE... ^

@bhsramz Welcome to the Rockstor community and thanks for helping to support Rockstor via a stable subscription.

Do you know if the update was from a much older version? The reason I ask is the error indicates a possible missing database entry:

but we haven’t changed our database for quite some time now, however the pincard field was added > 2 years ago.

@Flyer is this your reading of the error log? Pincard was added even before we changed from south to the django built migration mechanism, although it was towards the end of that run (46 of 47 total).

@phillxnet Thanks for the welcome. I began a 3 year Rockstor stable subscription on 3/23/17. I can’t imagine that the recent updates have been from much older versions. As a matter of fact, when I first noticed my Time Machine backup disk was not available, a week ago, the version displayed was 3.9.2-26. It has apparently updated again since then.

Could the problem be related to my OS disk? When I built my NAS I used an older 180GB HDD I had laying about for the Rockstor installation. Maybe that hard drive is failing. Any suggestions on how I could diagnose problems with the disk would be appreciated. Thanks!

@bhsramz

OK, so probably not a really old system that has recently been updated then.
Could you confirm the version via a paste of the following: just to make sure:

yum info rockstor

Thanks.

Always a possibility. It’s just pretty strange that it’s a database field error.

Your could try running a S.M.A.R.T self test, with that link showing the last tab on how you can do this via Rockstor’s Web-UI. Bad sectors are the thing to look out for mostly. You could also do a btrfs scrub of the system pool, often called rockstor_rockstor. Make sure you are actually on a stable channel release first though as the feedback on scrubs is much better.

Hope that helps and let us know how you get.

My Rockstor system is headless and I manage it through the web UI, so getting a paste of the output is not easy, but I can confirm it is now 3.9.2-31. I realize the error is a database error, but I was thinking bad sectors on the disk might cause them.

I was not able to run a S.M.A.R.T. test on the drive through the UI. I ran a btrfs scrub and rebooted the box, and now I am unable to load any of the pages. The screen shot below shows the “Unknown internal error” I get when trying to load any page. There is no traceback to provide either. Any thoughts on whether I should try to reinstall or install on a new drive?

Well, sorry I couldn’t help pinpoint the cause of my particular error. I was able to reinstall Rockstor on the same disk, and it has been working for about 2 weeks now. I was able to back up my Macs again and transfer an image from a Time Machine backup to a new machine.

It does seem less responsive than before, so I may replace the disk preemptively. I have a feeling that the +10 year-old disk is signaling it is ready for retirement. Thanks for your assistance @phillxnet!

@bhsramz Glad your re-install and import worked OK and thanks for the thanks and update; much appreciated.

No worries.

Sometimes failing disks can have lower performance as they may be busy re-trying actions due to internal errors.

Yes, possibly.

Let us know how it goes with that potentially suspect system drive.

You might find the following btrfs command usefull, we plan to integrate it into the Web-UI in time.

btrfs dev stats /mnt2/rockstor_rockstor

or you can use a disk name instead of a mount point.

The open issue for this is:

https://github.com/rockstor/rockstor-core/issues/1532

Hope that helps and remember to do some scrubs on that system disk from time to time.