Having trouble importing pool into Rockstor rebuild

[cjc1959au (Christopher Collins)]

[Please complete the below template with details of the problem reported on your Web-UI. Be as detailed as possible. Community members, including developers, shall try and help. Thanks for your time in reporting this issue! We recommend purchasing commercial support for expedited support directly from the developers.]

Brief description of the problem

Trying to import BTRFS pool after reinstallation

Detailed step by step instructions to reproduce the problem

Went to Storage → Disks and configured all disks as complete BTRFS. Did not do wipe.
Went to Storage → Pools - Tried to import first disk and error listed below came up

Web-UI screenshot

Error Traceback provided on the Web-UI

Traceback (most recent call last): File "/opt/rockstor/src/rockstor/storageadmin/views/disk.py", line 699, in _btrfs_disk_import do.save() File "/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/base.py", line 734, in save force_update=force_update, update_fields=update_fields) File "/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/base.py", line 762, in save_base updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields) File "/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/base.py", line 827, in _save_table forced_update) File "/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/base.py", line 877, in _do_update return filtered._update(values) > 0 File "/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/query.py", line 580, in _update return query.get_compiler(self.db).execute_sql(CURSOR) File "/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/sql/compiler.py", line 1062, in execute_sql cursor = super(SQLUpdateCompiler, self).execute_sql(result_type) File "/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/sql/compiler.py", line 840, in execute_sql cursor.execute(sql, params) File "/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/backends/utils.py", line 64, in execute return self.cursor.execute(sql, params) File "/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/utils.py", line 98, in __exit__ six.reraise(dj_exc_type, dj_exc_value, traceback) File "/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/backends/utils.py", line 64, in execute return self.cursor.execute(sql, params) OperationalError: deadlock detected DETAIL: Process 10722 waits for ShareLock on transaction 3548; blocked by process 10759. Process 10759 waits for ShareLock on transaction 3546; blocked by process 10722. HINT: See server log for query details.

[phillxnet (Philip Guyton)]

@cjc1959au Welcome to the Rockstor community.

Went to Storage -> Disks and configured all disks as complete BTRFS. Did not do wipe.

This step is not required when using whole disk as it’s the default. See Import whole disk BTRFS, ie it should identify it as containing a “whole disk BTRFS filesystem” without this step.

Went to Storage -> Pools - Tried to import first disk and error listed below came up

Presumably here you meant one of the import icons next to each disk of a former pool, as per above link.

The error you received looks like a db race condition of sorts where one part of the database is waiting for another: hence the ‘deadlock detected’ bit. There have been a number of improvements to the import code over time so could you please indicate the version of Rockstor you are using while running into this issue.

Hope that helps and once we know the version we can take it from there.

[cjc1959au (Christopher Collins)]

Import whole disk BTRFS

OK, so at the top of my Web-UI screen, I have 2 versions there:

[ROCKSTOR 3.9.1-0]
linux: 4.10.6-1.el7.elrepo.x86_64

It was a clean install, but I think I did the updates from the machine itself, rather than through the UI. Would that make a difference?

I’ve also tried to ask a question about the stable updates before, but it never got answered, so I’m just on hold for now.

Christopher

[phillxnet (Philip Guyton)]

@cjc1959au Thanks for the info:

@cjc1959au

[ROCKSTOR 3.9.1-0]
linux: 4.10.6-1.el7.elrepo.x86_64

The top number is the version of the installed rockstor rpm, see:

yum info rockstor

the second is the linux kernel version.

Both show that you have not yet selected an update channel as both channels have updates to both the rockstor package and a newer (4.12) kernel kernel. See System - Software Update, or the prompt that pops up when visiting the Dashboard. Both of these package updates should help with import, the rockstor package will help on our (Rockstor’s) end and the kernel will should help with it carrying a newer btrfs filesystem driver. The former is the main benefit re the import, the latter for general filesystem improvements.

Selecting an update channel add an additional repository from which newer rockstor and kernel packages can be drawn. Although our kernel package is copied directly from the elrepo folks via their elrepo-kernel channel and is their kernel-ml (for main line) variant. But we are currently behind on kernel updates which is another story (see our in progress openSUSE news on the forum).

So if you first select an update channel and then do the following in a terminal as the root user:

yum update

we can then see if there are any errors, if so please copy the output into your reply here. I suggest this method rather than via the Web-UI as we seem to be in very different time zones and this will provide more info than if the update is done via the Web-UI which, if there is a problem, will just fail with no info. So just in case I’m suggest you update at the terminal after having first selected your chosen update channel.

Please see the top post in the following thread for info on the current state of the channels:

3.9.2-# Stable Channel Changelog

Intro As this thread is starting retroactively in this now long release cycle I’ll first give some context and move to detailing only the most recent releases. The canonical reference for all code changes is in our open source GitHub repo: This is the ‘rockstor’ package origin. This does not include the Rock-on definition files. An attempt is made to keep GitHub Release / git tags in line with package release versions but due to a variety of reasons one can lag the other for short periods. …

Essentially the last testing channel release was the first stable channel release of 3.9.2-0 (which never got an associated iso) so it’s well worth trying first in your case. The stable channel now has over 40 more fixes since then and it’s associated subscription payment helps to sustain the project’s continued development but definitely worth trying testing first; assuming you didn’t disable quotas on that pool previously which is currently only supported in the stable channel code.

@cjc1959au

I’ve also tried to ask a question about the stable updates before, but it never got answered, so I’m just on hold for now.

That’s strange as under your current user of @cjc1959au I have only this thread for your entire forum post history. Lets circle back around to this one once we have your system updated and your pool imported. Although the above info/references have hopefully addressed you prior query.

Hope that helps and apologies for the slow response.

[cjc1959au (Christopher Collins)]

Hi again…

OK, so I have done a complete re-installation. I clean installed, then set “Testing Updates” so I could get 3.9.1-16, and then did a “yum update” in a command window.

Rebooted a couple of times to be able to ensure all was up to date.

Reinstalled my PERC-310 controller so that the storage disks are listed under lsblk and I can see all 6 * 8TB in the list.

Went to do the import of the BTRFS setup on the first of the six disks and ended up getting the following message:

##### Houston, we’ve had a problem.

Failed to import any pool on this device(2). Error: deadlock detected DETAIL: Process 7342 waits for ShareLock on transaction 3051; blocked by process 7379. Process 7379 waits for ShareLock on transaction 3049; blocked by process 7342. HINT: See server log for query details.

And underneath, it had this information:

       Traceback (most recent call last):

File “/opt/rockstor/src/rockstor/storageadmin/views/disk.py”, line 705, in _btrfs_disk_import
do.save()
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/base.py”, line 734, in save
force_update=force_update, update_fields=update_fields)
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/base.py”, line 762, in save_base
updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/base.py”, line 827, in _save_table
forced_update)
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/base.py”, line 877, in _do_update
return filtered._update(values) > 0
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/query.py”, line 580, in _update
return query.get_compiler(self.db).execute_sql(CURSOR)
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/sql/compiler.py”, line 1062, in execute_sql
cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/sql/compiler.py”, line 840, in execute_sql
cursor.execute(sql, params)
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/backends/utils.py”, line 64, in execute
return self.cursor.execute(sql, params)
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/utils.py”, line 98, in exit
six.reraise(dj_exc_type, dj_exc_value, traceback)
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/backends/utils.py”, line 64, in execute
return self.cursor.execute(sql, params)
OperationalError: deadlock detected
DETAIL: Process 7342 waits for ShareLock on transaction 3051; blocked by process 7379.
Process 7379 waits for ShareLock on transaction 3049; blocked by process 7342.
HINT: See server log for query details.

I know you asked about versions last time, so I’m adding those:

ROCKSTOR 3.9.1-16
Linux: 4.12.4-1.el7.elrepo.x86_64

It talks about a “Server Log” on the bottom of that information, but when I go to “Logs Manager”, I can’t find one called “Server Log”.

OK, so I just tried importing on the second of the six disks and it seems to have worked OK, I can now see rs01_pool01, but I really don’t know why.

Hopefully, this might help someone…

Thanks for your help so far!

[phillxnet (Philip Guyton)]

@cjc1959au Glad import worked for you, if only on the second attempt.

OK, so I just tried importing on the second of the six disks and it seems to have worked OK, I can now see rs01_pool01 …

re:

… but I really don’t know why.

These types of problems can be quite trickyhowever. But I’m pretty sure there have been further improvements in this area, during the current stable release run, that should have improved things further. Plus we have a pending pull request that should also help with simplifying our simultaneous db access.

Thanks for posting the update / your findings.