Houston, We've had a problem 12345

Support
1

Logged into Rockstor. no matter what page I am on, I see the following at the top of each page:

        Traceback (most recent call last):

File “/opt/rockstor/.venv/lib/python3.11/site-packages/django/db/backends/base/base.py”, line 313, in _commit
return self.connection.commit()
^^^^^^^^^^^^^^^^^^^^^^^^
File “/opt/rockstor/.venv/lib/python3.11/site-packages/psycopg/connection.py”, line 896, in commit
self.wait(self._commit_gen())
File “/opt/rockstor/.venv/lib/python3.11/site-packages/psycopg/connection.py”, line 969, in wait
return waiting.wait(gen, self.pgconn.socket, timeout=timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/opt/rockstor/.venv/lib/python3.11/site-packages/psycopg/waiting.py”, line 350, in wait_poll
s = gen.send(ready)
^^^^^^^^^^^^^^^
File “/opt/rockstor/.venv/lib/python3.11/site-packages/psycopg/connection.py”, line 538, in _commit_gen
yield from self._exec_command(b"COMMIT")
File “/opt/rockstor/.venv/lib/python3.11/site-packages/psycopg/connection.py”, line 472, in _exec_command
raise e.error_from_result(result, encoding=pgconn_encoding(self.pgconn))
psycopg.errors.ForeignKeyViolation: insert or update on table “storageadmin_networkdevice” violates foreign key constraint “storageadmin_network_connection_id_6a520cbf_fk_storagead”
DETAIL: Key (connection_id)=(4) is not present in table “storageadmin_networkconnection”.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File “/opt/rockstor/src/rockstor/rest_framework_custom/generic_view.py”, line 41, in _handle_exception
yield
File “/opt/rockstor/src/rockstor/storageadmin/views/network.py”, line 236, in get_queryset
self._refresh_devices()
File “/usr/lib64/python3.11/contextlib.py”, line 80, in inner
with self._recreate_cm():
File “/opt/rockstor/.venv/lib/python3.11/site-packages/django/db/transaction.py”, line 263, in exit
connection.commit()
File “/opt/rockstor/.venv/lib/python3.11/site-packages/django/utils/asyncio.py”, line 26, in inner
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File “/opt/rockstor/.venv/lib/python3.11/site-packages/django/db/backends/base/base.py”, line 337, in commit
self._commit()
File “/opt/rockstor/.venv/lib/python3.11/site-packages/django/db/backends/base/base.py”, line 312, in _commit
with debug_transaction(self, “COMMIT”), self.wrap_database_errors:
File “/opt/rockstor/.venv/lib/python3.11/site-packages/django/db/utils.py”, line 91, in exit
raise dj_exc_value.with_traceback(traceback) from exc_value
File “/opt/rockstor/.venv/lib/python3.11/site-packages/django/db/backends/base/base.py”, line 313, in _commit
return self.connection.commit()
^^^^^^^^^^^^^^^^^^^^^^^^
File “/opt/rockstor/.venv/lib/python3.11/site-packages/psycopg/connection.py”, line 896, in commit
self.wait(self._commit_gen())
File “/opt/rockstor/.venv/lib/python3.11/site-packages/psycopg/connection.py”, line 969, in wait
return waiting.wait(gen, self.pgconn.socket, timeout=timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “/opt/rockstor/.venv/lib/python3.11/site-packages/psycopg/waiting.py”, line 350, in wait_poll
s = gen.send(ready)
^^^^^^^^^^^^^^^
File “/opt/rockstor/.venv/lib/python3.11/site-packages/psycopg/connection.py”, line 538, in _commit_gen
yield from self._exec_command(b"COMMIT")
File “/opt/rockstor/.venv/lib/python3.11/site-packages/psycopg/connection.py”, line 472, in _exec_command
raise e.error_from_result(result, encoding=pgconn_encoding(self.pgconn))
django.db.utils.IntegrityError: insert or update on table “storageadmin_networkdevice” violates foreign key constraint “storageadmin_network_connection_id_6a520cbf_fk_storagead”
DETAIL: Key (connection_id)=(4) is not present in table “storageadmin_networkconnection”.

2

P.S. Rcokstor seems to be trying to run this command:

##### Houston, we’ve had a problem.

insert or update on table “storageadmin_networkdevice” violates foreign key constraint “storageadmin_network_connection_id_6a520cbf_fk_storagead” DETAIL: Key (connection_id)=(4) is not present in table “storageadmin_networkconnection”.

I also note there are no new snapshots since the 1st January 2025.

3

@Rocky12345 hello again.

Was there anything you’ve done recently in the networking space like changing its static IP address, or adding a new network device to the box (or are you using bonded interfaces)? Or installation of additional Rockons?

@Flox some time ago (and on an older version) pointed out some of the context on where this might be occurring, but also the difficulty to pinpoint what causes this situation:

Also, did you perform a reboot? In another instance of this error, it seemed to have fixed the issue.

2 Likes
4

Thanks a lot, @Hooverdan !

Yes, this is a tricky one to debug and may be some sort of race condition. Ideally, it would be great to have the current state of the related tables: storageadmin_networkconnection and storageadmin_networkdevice. This can be retrieved using psql. I’m unfortunately not with my machine to give you the exact command (@Hooverdan , would you, by any chance?).

The nmcli outputs could be helpful as well:
nmcli d show
and nmcli d show <devname> for each listed in the first command.

Sorry for the very short answer… I’ll try to give more details but that would be much later tonight, unfortunately.

1 Like
5

@Flox I assume you’re talking about this:

psql -U rocky -d storageadmin -c "SELECT * FROM storageadmin_networkconnection";

password will be rocky

Which should give you as an output a wide table with various entries, looking like this:

 id |        name        |                 uuid                 |   state   | autoconnect | ipv4_method | ipv4_addresses  |  ipv4_gw   |  ipv4_dns  | ipv4_dns_search | ipv6_method | ipv6_addresses | ipv6_gw | ipv6_dns | ipv6_dns_search | master_id                 
----+--------------------+--------------------------------------+-----------+-------------+-------------+-----------------+------------+------------+-----------------+-------------+----------------+---------+----------+-----------------+-----------                
 20 | lo                 | 7686c148-820f-4f29-8cfa-47b40aee2ef6 | activated | t           | manual      | 127.0.0.1/8     |            |            |                 |             |                |         |          |                 |                           
 14 | docker0            | d862a23d-79bb-4ee1-99ad-702468b7b1bb | activated | t           | manual      | 172.17.0.1/16   |            |            |                 |             |                |         |          |                 |                           
  2 | Wired connection 2 | 499db57f-cd2c-156b-bbac-6c64277f0063 |           | t           | auto        |                 |            |            |                 |             |                |         |          |                 |                           
  1 | Wired connection 1 | def6efd2-c0ba-32a6-a213-f74529b98dd5 | activated | t           | auto        | 172.19.0.199/24 | 172.19.0.1 | 172.19.0.1 |                 |             |                |         |          |                 |                           
(4 rows)

And for the other table @Flox requested:

psql -U rocky -d storageadmin -c "SELECT * FROM storageadmin_networkdevice";
2 Likes
6

don’t know if this is relevant, but I am running Rockstor on ProxMox.

I have since rebooted and it seems to have cleared the issue,

7 
 id |        name        |                 uuid                 |   state   | autoconnect | ipv4_method |  ipv4_addresses  |   ipv4_gw   |   ipv4_dns   | ipv4_dns_search | ipv6_method | ipv6_addresses | ipv6_gw | ipv6_dns | ipv6_dns_search | master_id 
----+--------------------+--------------------------------------+-----------+-------------+-------------+------------------+-------------+--------------+-----------------+-------------+----------------+---------+----------+-----------------+-----------
  8 | docker0            | c4b74e67-5de8-42de-ba6f-732614b7e1c9 | activated | t           | manual      | 172.17.0.1/16    |             |              |                 |             |                |         |          |                 |          
  7 | lo                 | f3bd8813-8fa0-4463-a511-54a7821542df | activated | t           | manual      | 127.0.0.1/8      |             |              |                 |             |                |         |          |                 |          
  1 | Wired connection 1 | 934c97bb-04d4-3ae5-9c79-fa7c79b9c02f | activated | t           | auto        | 192.168.0.187/24 | 192.168.0.1 | 192.168.0.29 |                 |             |                |         |          |                 |          
(3 rows)
8 
admin@Rocky:~> psql -U rocky -d storageadmin -c "SELECT * FROM storageadmin_networkdevice";                                                    
Password for user rocky:                                                                                                                       
 id |  name   |  dtype   |        mac        |            state             |  mtu  | connection_id                                            
----+---------+----------+-------------------+------------------------------+-------+---------------                                           
  3 | docker0 | bridge   | 02:42:0C:FE:7C:9A | 100 (connected (externally)) | 1500  |             8                                            
  2 | lo      | loopback | 00:00:00:00:00:00 | 100 (connected (externally)) | 65536 |             7                                            
  1 | eth0    | ethernet | 92:4D:6C:B9:BF:5D | 100 (connected)              | 1500  |             1                                            
(3 rows)