@djevo1 just had an upgrade from old Rockstor to 3.9.0-1 and it seems ok, so not helping.
Can you help us with some checks? 
From a shell type this : psql -U rocky -d storageadmin -c "\d storageadmin_share"
When asked for password : rocky (Rockstor default values)
Please provide output with table schema (first part)
Thanks
Mirko
Table "public.storageadmin_share"
Column | Type | Modifi
ers
------------------+--------------------------+----------------------------------
-------------------------------
id | integer | not null default nextval('storage
admin_share_id_seq'::regclass)
pool_id | integer | not null
qgroup | character varying(100) | not null
name | character varying(4096) | not null
uuid | character varying(100) |
size | bigint | not null
owner | character varying(4096) | not null
group | character varying(4096) | not null
perms | character varying(9) | not null
toc | timestamp with time zone | not null
subvol_name | character varying(4096) | not null
replica | boolean | not null
compression_algo | character varying(1024) |
rusage | bigint | not null
eusage | bigint | not null
pqgroup | character varying(32) | not null
pqgroup_eusage | bigint | not null
pqgroup_rusage | bigint | not nullOk, storageadmin_share.pqgroup_rusage does not exist…but exists!, so Django migration had new storageadmin_share new fields, but we still fail 
Who wants to be a millionaire mode on, I “phone a friend” 
: @suman any idea about this ? Seems like a kind of “ghost db migration” ?!?
Some guessing starting from @djevo1 log:
we have un update, we initrock and then this failing:
[23/Mar/2017 08:50:23] ERROR [storageadmin.middleware:32] Exception occured while processing a request. Path: /api/commands/bootstrap method: POST [23/Mar/2017 08:50:23] ERROR [storageadmin.middleware:33] column storageadmin_share.pqgroup_rusage does not exist LINE 1: ...n_share"."rusage", "storageadmin_share"."eusage", "storagead... ^ Traceback (most recent call last): File "/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/core/handlers/base.py", line 132, in get_response response = wrapped_callback(request, *callback_args, **callback_kwargs) [etc etc etc]
and finally this
File "/opt/rockstor/src/rockstor/storageadmin/views/command.py", line 90, in post import_shares(p, request) File "/opt/rockstor/src/rockstor/storageadmin/views/share_helpers.py", line 73, in import_shares shares = [s.name for s in Share.objects.filter(pool=pool)] File "/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/query.py", line 162, in __iter__ self._fetch_all() File "/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/query.py", line 965, in _fetch_all self._result_cache = list(self.iterator()) File "/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/query.py", line 238, in iterator results = compiler.execute_sql() File "/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/sql/compiler.py", line 840, in execute_sql cursor.execute(sql, params) File "/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/backends/utils.py", line 64, in execute return self.cursor.execute(sql, params) File "/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/utils.py", line 98, in __exit__ six.reraise(dj_exc_type, dj_exc_value, traceback) File "/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/backends/utils.py", line 64, in execute return self.cursor.execute(sql, params) ProgrammingError: column storageadmin_share.pqgroup_rusage does not exist LINE 1: ...n_share"."rusage", "storageadmin_share"."eusage", "storagead... ^
To @djevo1, can you try this from a shell?
psql -U rocky -d storageadmin -c "SELECT * FROM django_migrations;"
(same rocky pass) - Thanks
Mirko
id | app | name | applied
----+-----------------+---------------------------------------+-------------------------------
1 | contenttypes | 0001_initial | 2017-03-23 06:14:44.103263-07
2 | auth | 0001_initial | 2017-03-23 06:14:44.124905-07
3 | oauth2_provider | 0001_initial | 2017-03-23 06:14:44.155001-07
4 | storageadmin | 0001_initial | 2017-03-23 06:14:44.179394-07
5 | storageadmin | 0002_auto_20161125_0051 | 2017-03-23 06:14:47.579102-07
6 | storageadmin | 0003_auto_20170114_1332 | 2017-03-23 06:14:52.97075-07
7 | contenttypes | 0002_remove_content_type_name | 2017-03-23 06:15:10.426857-07
8 | auth | 0002_alter_permission_name_max_length | 2017-03-23 06:15:10.512811-07
9 | auth | 0003_alter_user_email_max_length | 2017-03-23 06:15:10.575783-07
10 | auth | 0004_alter_user_username_opts | 2017-03-23 06:15:10.622553-07
11 | auth | 0005_alter_user_last_login_null | 2017-03-23 06:15:10.830232-07
12 | auth | 0006_require_contenttypes_0002 | 2017-03-23 06:15:10.855935-07
(12 rows)@phillxnet & @suman checking djevo1 django_migrations table over storageadmin & previous info we can say :
pqgroup_rusage/e_usage
0003_auto_20170114_1332 over migration table)…and it should be all fine.
I think it’s more a “weirdo” with some special conditions rather then a pure coding error.
@djevo1 once again thanks for your help, can you provide other outputs?
echo -e 'select * from storageadmin_share; \n select * from storageadmin_pool;' | psql -U rocky -d storageadmin same rocky pass
we echo two sql statements to grab share and pool data from db, psql not supporting multiple command with -c param so we use a pipe
Mirko
id | pool_id | qgroup | name | uuid | size | owner | group | perms | toc | subvol_name | replica | compression_algo | rusage | eusage | pqgroup | pqgroup_eusage | pqgroup_rusage
----+---------+--------+------+------+------+-------+-------+-------+-----+-------------+---------+------------------+--------+--------+---------+----------------+----------------
(0 rows)
id | name | uuid | size | raid | toc | compression | mnt_options | role
----+------+------+------+------+-----+-------------+-------------+------
(0 rows)?? No pools and no shares?
I have a single pool, and two shares.
@suman :
but db is empty/seems empty?!? see previous message from @djevo1
@djevo1 do you see your pool and your shares via WebUI?
M.
I can see the shares available via afp, but they do not lead to the pool. If I ssh into the box I can find the pool via the mnt2 directory. As stated above I have not been able to access the web-ui.
Thanks @djevo1, did not catch it
Your infos help me on debugging, will try to reproduce it by flushing all pools and shares (my suspicion it’s a missing check when system has no pools/shares)
Mirko
Note to @suman & @phillxnet : any idea how we can have no pools & no shares with Rockstor defaults too? (I mean home & root shares plus rockstor default pool)
M.
Hi @djevo1 your no pools no shares is an odd, so had some tests on this.
First one, flushing all shares from PostgreSQL db : our Rockstor is supposed to be self-healing, so on every share rescan we restore our data. First time I test it with an “hard delete” and it’s fine, restoring system shares like root and home too
Next step, testing: flushing pools too
M.
@djevo1 asking other infos, can you provide this? :
psql -U rocky -d storageadmin -c "SELECT * FROM storageadmin_disk;" (pass always rocky)
Let me explain : shares require pools, pools require disks.
You can force a share delete (bot /home and /root although you don’t have them), you can delete a pool, but not rockstor default rockstor_rockstor pool…but you don’t have it!
My idea : probably we’re missing a check raising those WebUI errors, but being this related to a missing rockstor_rockstor pool I think you had a “freaky update” (ex. blocked/partially blocked during yum update over Rockstor package and others)
M.
id | pool_id | name | size | offline | parted | btrfs_uuid | model | serial | transport | vendor | s
mart_available | smart_enabled | smart_options | role
----+---------+------+------+---------+--------+------------+-------+--------+-----------+--------+--
---------------+---------------+---------------+------
(0 rows)Hi @djevo1, your last post confirms my idea about a “broken system” :
you don’t have shares, you don’t have pools and you don’t have disks too
@suman & @phillxnet any idea about this? I don’t think this can not be because of a Rockstor update
Mirko
These problems have happened over and over even after a complete resintall.
@djevo1 Could you give some info about the nature of your install, ie did you select “Encrypt my data” during install (which can lead to no pools or shares) or do you have any ‘custom’ config that might be at the root cause of your apparently perpetual problems:
It might also be useful to know what the hardware arrangement is, ie do you use a flash drive for you system install and if so what is the model and size, and has the system disk hardware changed over time? Also have your tested your memory integrity etc? See our Pre-Install Best Practice (PBP).
Do you have a uefi install?
Just trying to get more context as the symptoms you described are indicative of multiple simultaneous issues but the known db update one is not known to actually stop the UI, which you also report. Hence investigating corruption causes (@Flyer’s “broken system”) with your report of repeated instances even after re-install leads me to seek other causes.
What is the output of for example:
btrfs fi show
and
btrfs fi usage /mnt2/rockstor_rockstor
and
ls -la /dev/disk/by-id/
Apologies if I’ve misread this thread but just trying to chip in.
Also note that we now have a 3.9.0.iso which circumvents the know intermittent update issues we saw half way through the 3.8 testing channel releases so do use this version if you end up reinstalling.
No UEFI install.
No encrypt my data
Standard install
Boot disk made from ISO, installed on a recommended sony flashdrive
BTRFS fi output
Label: 'rockstor_rockstor' uuid: 713f423e-ed9d-4cfd-b8b1-fd76528e870e
Total devices 1 FS bytes used 2.90GiB
devid 1 size 6.04GiB used 5.51GiB path /dev/sdg3
Label: 'PrimaryR5' uuid: 52ff3397-57dc-4dc0-b282-61d530fd515e
Total devices 6 FS bytes used 2.39TiB
devid 1 size 1.82TiB used 616.02GiB path /dev/sdb
devid 2 size 1.82TiB used 616.02GiB path /dev/sdd
devid 3 size 1.82TiB used 616.02GiB path /dev/sda
devid 4 size 1.82TiB used 616.02GiB path /dev/sdf
devid 5 size 1.82TiB used 616.02GiB path /dev/sde
devid 6 size 1.82TiB used 616.02GiB path /dev/sdc
btrfs fi usage
Overall:
Device size: 6.04GiB
Device allocated: 5.51GiB
Device unallocated: 534.00MiB
Device missing: 0.00B
Used: 2.97GiB
Free (estimated): 2.56GiB (min: 2.30GiB)
Data ratio: 1.00
Metadata ratio: 1.97
Global reserve: 16.00MiB (used: 0.00B)
Data,single: Size:4.88GiB, Used:2.84GiB
/dev/sdg3 4.88GiB
Metadata,single: Size:8.00MiB, Used:0.00B
/dev/sdg3 8.00MiB
Metadata,DUP: Size:309.00MiB, Used:63.47MiB
/dev/sdg3 618.00MiB
System,single: Size:4.00MiB, Used:0.00B
/dev/sdg3 4.00MiB
System,DUP: Size:8.00MiB, Used:16.00KiB
/dev/sdg3 16.00MiB
Unallocated:
/dev/sdg3 534.00MiB
ls -la /dev/disk/by-id/
drwxr-xr-x 2 root root 360 Mar 23 08:17 .
drwxr-xr-x 6 root root 120 Mar 23 08:17 ..
lrwxrwxrwx 1 root root 9 Mar 23 08:17 ata-HGST_HDS724020ALE640_PK2111P6KR0REV -> ../../sdc
lrwxrwxrwx 1 root root 9 Mar 23 08:17 ata-Hitachi_HDS722020ALA330_JK1101BFH5BA2F -> ../../sde
lrwxrwxrwx 1 root root 9 Mar 23 08:17 ata-Hitachi_HDS722020ALA330_JK1105B8G5YXDX -> ../../sdb
lrwxrwxrwx 1 root root 9 Mar 23 08:17 ata-Hitachi_HDS722020ALA330_JK11A8BFHSVNSF -> ../../sdf
lrwxrwxrwx 1 root root 9 Mar 23 08:17 ata-Hitachi_HDS722020ALA330_JK11B1BFHTZRKF -> ../../sdd
lrwxrwxrwx 1 root root 9 Mar 23 08:17 ata-Hitachi_HDS722020ALA330_JK11B1BFHU3VHF -> ../../sda
lrwxrwxrwx 1 root root 9 Mar 23 08:17 usb-Sony_Storage_Media_7C071058A21AA21D64-0:0 -> ../../sdg
lrwxrwxrwx 1 root root 10 Mar 23 08:17 usb-Sony_Storage_Media_7C071058A21AA21D64-0:0-part1 -> ../../sdg1
lrwxrwxrwx 1 root root 10 Mar 23 08:17 usb-Sony_Storage_Media_7C071058A21AA21D64-0:0-part2 -> ../../sdg2
lrwxrwxrwx 1 root root 10 Mar 23 08:17 usb-Sony_Storage_Media_7C071058A21AA21D64-0:0-part3 -> ../../sdg3
lrwxrwxrwx 1 root root 9 Mar 23 08:17 wwn-0x5000cca222c2b628 -> ../../sdb
lrwxrwxrwx 1 root root 9 Mar 23 08:17 wwn-0x5000cca229d087df -> ../../sde
lrwxrwxrwx 1 root root 9 Mar 23 08:17 wwn-0x5000cca229d8f027 -> ../../sdf
lrwxrwxrwx 1 root root 9 Mar 23 08:17 wwn-0x5000cca229d973c2 -> ../../sdd
lrwxrwxrwx 1 root root 9 Mar 23 08:17 wwn-0x5000cca229d98340 -> ../../sda
lrwxrwxrwx 1 root root 9 Mar 23 08:17 wwn-0x5000cca22df44964 -> ../../sdc