Column storageadmin_share.pqgroup_rusage does not existLINE 1: ...n_share"."rusage", "storageadmin_share"."eusage", "storagead... ^

Brief description of the problem

New install after update and reboot reports the following error:
column storageadmin_share.pqgroup_rusage does not exist LINE 1: …n_share".“rusage”, “storageadmin_share”.“eusage”, "storagead… ^

Detailed step by step instructions to reproduce the problem

With installation I had the same problem as it is described here:

I installed to a different system, but the symptom was the same and with ver3.8.15 the install was OK.
I just followed the advice to update to latest version.
Activated testing updates and run the update what was successful.
When server came back it reported kernel version mismatch, so I did reboot.
After reboot I got this message below.
Creating pool was OK, but could not create a share, the same message was thrown.
I repeated the install to another disk (just to retry) but got the same result.

Web-UI screenshot

Error Traceback provided on the Web-UI

        Traceback (most recent call last):

File “/opt/rockstor/src/rockstor/rest_framework_custom/generic_view.py”, line 41, in _handle_exception
yield
File “/opt/rockstor/src/rockstor/storageadmin/views/share.py”, line 100, in get_queryset
reverse=reverse)
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/query.py”, line 162, in iter
self._fetch_all()
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/query.py”, line 965, in _fetch_all
self._result_cache = list(self.iterator())
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/query.py”, line 238, in iterator
results = compiler.execute_sql()
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/sql/compiler.py”, line 840, in execute_sql
cursor.execute(sql, params)
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/backends/utils.py”, line 64, in execute
return self.cursor.execute(sql, params)
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/utils.py”, line 98, in exit
six.reraise(dj_exc_type, dj_exc_value, traceback)
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/backends/utils.py”, line 64, in execute
return self.cursor.execute(sql, params)
ProgrammingError: column storageadmin_share.pqgroup_rusage does not exist
LINE 1: …n_share".“rusage”, “storageadmin_share”.“eusage”, "storagead…
^

@zgyivi First off, welcome to the Rockstor community forum and thanks for reporting your findings.

I’m pretty sure this is related to a similar report we received yesterday so linking to that report from this thread:
Null value in column “pqgroup_eusage” violates not-null constraintDETAIL: Failing row contains (6, 4, 0/259, data1, null, 4831838208, root, root, 755, 2017-02-14 17:12:34.201143 00, data1, f, no, 0, 0, 2015/4, null, null)

It could be that this is related to some recent share size calculation changes and I believe the intrepid @Flyer is gathering ‘angles’, hence my link.

Thanks again for your detailed report.

You got it! @phillxnet did you find similar issues on your envs ?? (currently my working env is dead, had to kill it and others to free & re-order Proxmox disks space :disappointed:) - Asking @suman help too, that Django migration worked well on dev env and not able to reproduce this
M.

migrations got applied correctly in our staging and prod boxes here too. I have noticed an oddity in the past, very rarely, during rpm update, some migration fails to apply. I was never able to reproduce it on demand to debug further. Do we have reproducible steps on this one?

@zgyivi We just released 3.8.16-12. Could you try and update? This will give another opportunity to apply the same migration and might work this time.

@suman I have just tried the update (3.8.16-12.), and it seems to work. Boot OK, and no error message. I will try to set shares shortly…

@suman I have tried a new install, and as 3.8.16-1 iso is not working on my system, I used 3.8.15 and then I updated to 3.8.16-12 directly.
I got the same error message as with 3.8.16-11 earlier.
“Column storageadmin_share.pqgroup_rusage does not existLINE 1: …”

Then I tried yum update on the consol an it reported 2 pending updates:
python ver:2.7.5-48.el7 and python-libs ver:2.7.5-48.el7
I let them get updated, but it did not solve the problem.

Then I tried yum reinstall rockstore and it fixed the system.
After reboot it come up without the error message.

Possibly the python update has no relevance and reinstalling rockstor would have helped anyway.
Hope this helps to find the root cause…

2 Likes

Thanks for your feedback @zgyivi :slight_smile:

Can we mark this solved?? @suman and @phillxnet I think we anyway need to investigate updates issue

Mirko

@Flyer @suman Actually it is problem again with 3.8.16-13.
After I update to the latest version I got the same message.
I tried a new install on a different hardware but it is exactly the same.
Even yum reinstall rockstor does not help now.

Zoltan

Downgrading (yum downgrade rockstor) to 3.8.16-12 fixed the problem.

Hi @zgyivi,
did you have also an update after downgrade?
Mirko

Assuming you are on 3.8.16-12 now, Could you capture the entire output of yum update rockstor? This output will have clues, if the problem appears again.

Update from 3.8.16-12 to 3.8.16-13 is OK.
Sorry @suman I did not capture the update output, as I just read your post, it did look smooth, no unusual message and I did not get the error message after reboot.
Previously I updated from 3.8.15 to 3.8.16-13 directly, that caused the error.
It seems you have a some step in 3.8.16-12 release what is fixing the database, but it is not included in 3.8.16-13.
Zoltan

Hi.
I just had the same issue as zgyivi, with a similar upgrade path (3.8.15 -> 3.8.16-14). Downgrading to -12 and then upgrading to -14 again worked for me.
@suman I’m going to set up a similar server next week and will try to capture the output from the 3.8.15 -> 3.8.16-14. Let me know whether I need to capture anything extra.

I’ve been having the same issue too. Just updated to 3.8.16-15 and had the same error for the column. I updated to 3.8.16-15 and now I have the following stacktrace

            Traceback (most recent call last):
  File "/opt/rockstor/src/rockstor/rest_framework_custom/generic_view.py", line 41, in _handle_exception
    yield
  File "/opt/rockstor/src/rockstor/storageadmin/views/user.py", line 106, in get_queryset
    return combined_users()
  File "/opt/rockstor/src/rockstor/storageadmin/views/ug_helpers.py", line 69, in combined_users
    temp_uo.pincard_allowed, temp_uo.has_pincard = pincard_states(temp_uo)  # noqa E501
  File "/opt/rockstor/src/rockstor/system/pinmanager.py", line 146, in pincard_states
    pincard_present = has_pincard(user)
  File "/opt/rockstor/src/rockstor/system/pinmanager.py", line 129, in has_pincard
    pins = Pincard.objects.filter(user=int(uid_field)).count()
  File "/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/query.py", line 318, in count
    return self.query.get_count(using=self.db)
  File "/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/sql/query.py", line 466, in get_count
    number = obj.get_aggregation(using, ['__count'])['__count']
  File "/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/sql/query.py", line 447, in get_aggregation
    result = compiler.execute_sql(SINGLE)
  File "/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/sql/compiler.py", line 840, in execute_sql
    cursor.execute(sql, params)
  File "/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/utils.py", line 98, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
ProgrammingError: relation "storageadmin_pincard" does not exist
LINE 1: SELECT COUNT(*) AS "__count" FROM "storageadmin_pincard" WHE...

I’m trying yum downgrade rockstor right now and capturing the output. Just want to help fix this bug :frowning:

@gkadillak Hello again.
Are you sure your report is the same error as in this issues we have:

where as in your log we have:

which looks more like:

and

which were caused by a db migration tool change that meant no direct updates from Rockstor versions prior to 3.8.15, ie you had to install 3.8.15 as an interim step. However this newer issue has been seen (though rarely) when updated from rockstor-3.8.16-10.x86_64 to rockstor-3.8.16-12.x86_64.

That is in:
EDIT: corrected to link to 3.8.16-11 (not -10)

@Flyer added the issue specific db field in:

via:
commit 70ce761574f71be98b1a3df8f2f7a048ffc453b2
Share Usage reporting right sizes by MFlyer · Pull Request #1625 · rockstor/rockstor-core · GitHub
I’m currently looking into this but it seems initially that this is the db migration that fails to apply in some cases, resulting in the report subject of this thread.

I may have this all wrong so do share any findings corrections as it would be great to get this one sorted.

Sorry folks, linked to wrong release version. Have now edited that last post to point to:

for when
pqgroup_eusage and pqgroup_rusage were added.

Thanks @phillxnet :slight_smile:

Hi @suman . There is no warnings or anything strange in the output of the update. I can upload the log files somewhere if you want it, but I don’t think there is anything valuable in them, as they don’t show any kind of problem.

Just dropping by to confirm that this issue still exists. I did an update from 3.8.15 to 3.8.16-16 and ended up with the error described above. I rolled back, one version at a time, using yum downgrade rockstor and the error resolved itself when I got to 3.8.16-12.

So glad this thread exists! Thanks guys.

2 Likes