Missing DB table, causing an error on every page

malcolmrmclean · May 31, 2016, 7:43am

On every page of the rockstor UI, I get the following error:

Unknown internal error doing a GET to /api/appliances?page=1&format=json&page_size=15&count=

From the error logs, I can see the following:

ERROR [storageadmin.middleware:36] relation "storageadmin_networkinterface" does not exist
LINE 1: ...pe", "storageadmin_networkinterface"."state" FROM "storagead...

What would have caused the DB schema to not stay up to date with each update, and how can I fix this specific issue?

Parts of the site are still somewhat usable, but the majority isn’t.

Flyer · May 31, 2016, 7:57am

Good morning @malcolmrmclean and welcome to Rockstor Community!!

Can you please tell us your Rockstor version? this is the first step to understand your error

thanks
Flyer/Mirko

malcolmrmclean · May 31, 2016, 8:51am

The interface isn’t giving any version info, both the dashboard and software update pages only display this error.

/etc/rockstor-release says:

Rockstor release 3.8.10 (Core)

But rpm says 3.8.13 is installed:

rpm -qa | grep 'rockstor-'
rockstor-release-3-8.10.el7.x86_64
rockstor-3.8-13.x86_64

Auto-update is enabled.

Flyer · May 31, 2016, 9:01am

You’re on the stable branch, am I right?

I think we need @suman too, because this seems related to networkinterface moved to networkdevice + networkconnection (new bonding/teaming feature actually working fine over testing branch v. 3.8.13.12)

Looking around probably I’ve found a similar matter, please refer to this too → Announcing Rockstor 3.8-13

malcolmrmclean · May 31, 2016, 9:16am

Yes, I’m on the stable branch.

The post you refer to references errors in /api/rockons, whereas mine is in /api/appliances. I also don’t have any rock-ons installed, nor is it even switched on.

Running nmcli c show doesn’t show any network interfaces with duplicate names.

suman · May 31, 2016, 2:30pm

Thanks for helping out @Flyer.

@malcolmrmclean Are you by chance running Rockstor off USB? Do you mind sharing your system specs, I am just curious.

I have seen rare reports like this and it always turned out to be a migration failure on Rockstor running on USB. I’ve just created an issue to fix this properly. Please see the issue where I’ve also listed steps to manually fix the problem. Please do capture all the output and errors as you troubleshoot this and share them with us. Thanks!

Flox · May 31, 2016, 4:44pm

Hello!
I just wanted to share my experience as the mention of a Rockstor install on USB reminded me that I had a similar problem when my instance automatically updated to the latest stable release. In my case, I was still able to use the webUI to reboot Rockstor and then everything was fixed.
As @suman_chakravartula mentioned, I am running Rockstor off a USB flashdrive. It happened when the latest stable version was released so I’m not sure there is still useful information somewhere on my system, but if there’s anything I can share to help pinpoint a problem, I would be happy to help.

malcolmrmclean · June 1, 2016, 8:22am

It seems all it needed was a reboot. Marginally annoying, but working now at least.

malcolmrmclean · June 1, 2016, 9:54am

@suman No I’m not running rockstor off a USB, but rather a SSD.

rockmar · November 28, 2016, 11:58pm

I have just performed a fresh install from 3.8.12 then upgraded to the latest test build - 3.8.15-11 and am seeing a similar error on every page. I tried running the migration steps, both of them exited, essentially saying that relation already exists.
Running on HP N40L from 16GB USB stick.

Actual error for me is this:

Hostname: rs Mgmt IP: 192.168.0.106 Local Time: 23:55 (+0000 GMT) System Shell Uptime: 00:19 Linux: 4.8.7-1.el7.elrepo.x86_64

Houston, we’ve had a problem.

relation “storageadmin_networkdevice” does not exist LINE 1: …“state”, “storageadmin_networkdevice”.“mtu” FROM "storagead… ^

        Traceback (most recent call last):

File “/opt/rockstor/src/rockstor/rest_framework_custom/generic_view.py”, line 40, in _handle_exception
yield
File “/opt/rockstor/src/rockstor/storageadmin/views/network.py”, line 174, in get_queryset
self._refresh_devices()
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/utils/decorators.py”, line 145, in inner
return func(*args, **kwargs)
File “/opt/rockstor/src/rockstor/storageadmin/views/network.py”, line 154, in _refresh_devices
for ndo in NetworkDevice.objects.all():
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/query.py”, line 162, in iter
self._fetch_all()
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/query.py”, line 965, in _fetch_all
self._result_cache = list(self.iterator())
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/query.py”, line 238, in iterator
results = compiler.execute_sql()
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/sql/compiler.py”, line 840, in execute_sql
cursor.execute(sql, params)
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/backends/utils.py”, line 64, in execute
return self.cursor.execute(sql, params)
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/utils.py”, line 98, in exit
six.reraise(dj_exc_type, dj_exc_value, traceback)
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/backends/utils.py”, line 64, in execute
return self.cursor.execute(sql, params)
ProgrammingError: relation “storageadmin_networkdevice” does not exist
LINE 1: …“state”, “storageadmin_networkdevice”.“mtu” FROM "storagead…
^
`

Flyer · November 29, 2016, 10:58am

Hi @rockmar and welcome to Rockstor.

Without reading all your message, focus on this:

Why ?!?
Well, we have a 3.8.15 iso ready to download on sourceforge, so I don’t understand having 3.8.12 then upgrading

Next step:

What do you mean with “migration steps”?? Hope you’re talking about Rockstor updates and not Django migrations (users not supposed to manually do that)

My idea: sometime after a Rockstor update you need a reboot, if you performed 3 cycles updates (3.8.12 to 3.8.15-11 without having a reboot) probably that made something fail

Once again, the right way to install Rockstor: download latest iso then perform updates!

Hope you’ll solve, if not: come back here and we’ll help you!
M.

phillxnet · November 29, 2016, 12:45pm

@rockmar As @Flyer and @malcolmrmclean earlier in this thread mention, many updates depend on a reboot to ensure it’s the newer code you are running. Not sure but as you point out:

"storageadmin_networkdevice" does not exist

This looks like an expected db entry is no longer present. Between 3.8.12 and 3.8-15-11 there were many db changes, including in the network subsystem.

It would be useful to know if you can re-produce this error after a reboot, as it may be that the db changes specific to 3.8.15-11 have also affected you here as well as in your other forum post.

Thanks.

rockmar · November 29, 2016, 2:15pm

Why? Because 3.8.15-11 wouldn’t let me create a storage pool, so I went back to the version I had working previously to allow me to create them. Sorry for not being clear on the ‘migration’ steps, they are the ones referenced in Suman’s post on 31st May where it was suggested there is a fix, that did not work for me.

Yes, I have rebooted several times and still get the error.

Tomasz_Kusmierz · November 30, 2016, 1:56am

I’ve had a similar issues that were cured by simple reboot of rockstor:

rockmar · December 4, 2016, 5:24pm

For info I have rebooted many times, including after the update to 3.8.15-14 and still get the same error on every page. If I go to the users page then that page doesn’t load properly but the table is different:

ProgrammingError: relation “storageadmin_pincard” does not exist
LINE 1: SELECT COUNT(*) AS “__count” FROM “storageadmin_pincard” WHE…

Tomasz_Kusmierz · December 5, 2016, 1:55am

That’s ok I was more of promting that there were related issues solved by reboot - which might indicate an upgrade that went wrong on your system (did not upgrade certain tables)

Mahmoud87 · December 5, 2016, 10:37am

Hello @Tomasz_Kusmierz
I am getting the same error, after successful installation 3.8.15-10 and upgraded to -14 fine. I get mine in a different window though.

I did few reboots still pops up every now and then after few clicks on ok it disappears. Do you have any suggesting I can try? Thanks

Flyer · December 5, 2016, 2:30pm

Hi @Mahmoud87,
had a check over that page (samba shares) and on my side it’s ok (no datatables errors)

williams.tlr · December 6, 2016, 10:52pm

I’m seeing this as well on the Storage > Samba page. Rockstor 3.8.15-14. No users have pincard setup. Output from /opt/rockstor/var/log/rockstor.log below. This begins when I access the Samba page and the follows to every page I navigate to afterwards. Rebooting multiple times has not resolved the issue for me.

[06/Dec/2016 15:44:41] ERROR [storageadmin.util:47] exception: relation “storageadmin_pincard” does not exist
LINE 1: SELECT COUNT() AS “__count” FROM “storageadmin_pincard” WHE…
^
Traceback (most recent call last):
File “/opt/rockstor/src/rockstor/rest_framework_custom/generic_view.py”, line 40, in _handle_exception
yield
File “/opt/rockstor/src/rockstor/storageadmin/views/user.py”, line 105, in get_queryset
return combined_users()
File “/opt/rockstor/src/rockstor/storageadmin/views/ug_helpers.py”, line 69, in combined_users
temp_uo.pincard_allowed, temp_uo.has_pincard = pincard_states(temp_uo)
File “/opt/rockstor/src/rockstor/system/pinmanager.py”, line 136, in pincard_states
pincard_present = has_pincard(user)
File “/opt/rockstor/src/rockstor/system/pinmanager.py”, line 121, in has_pincard
pins = Pincard.objects.filter(user=int(uid_field)).count()
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/query.py”, line 318, in count
return self.query.get_count(using=self.db)
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/sql/query.py”, line 466, in get_count
number = obj.get_aggregation(using, [’__count’])[’__count’]
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/sql/query.py”, line 447, in get_aggregation
result = compiler.execute_sql(SINGLE)
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/sql/compiler.py”, line 840, in execute_sql
cursor.execute(sql, params)
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/backends/utils.py”, line 64, in execute
return self.cursor.execute(sql, params)
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/utils.py”, line 98, in exit
six.reraise(dj_exc_type, dj_exc_value, traceback)
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/backends/utils.py”, line 64, in execute
return self.cursor.execute(sql, params)
ProgrammingError: relation “storageadmin_pincard” does not exist
LINE 1: SELECT COUNT() AS “__count” FROM “storageadmin_pincard” WHE…

Mahmoud87 · December 6, 2016, 10:54pm

@williams.tlr I was going to write the same exact thing I am doing my final reboot and will see how it will go.
Note: It also happens when you go to the disks page as well as the samba share page.