So, I was finally able to patch my sender machines files locally with the contents of the linked PRs. I also validated that only those changes had been made.
Sadly the replication still fails, buuuut I get more logs. (somehow - I attribute this to the patched files that seemed to fix some things.)
Sender
[15/Aug/2026 20:10:04] INFO [smart_manager.replication.sender:341] Id: 03d502e0-045e-05a2-3d06-950700080009-14. Sending full replica: /mnt2/docker/.snapshots/docker_bazarr_config/docker_bazarr_config_14_replication_1
[15/Aug/2026 20:11:02] ERROR [smart_manager.data_collector:1017] Failed to update disk state.. exception: Exception while setting access_token for url(http://127.0.0.1:8000): HTTPConnectionPool(host='127.0.0.1', port=8000): Read timed out. (read timeout=2). content: None
[15/Aug/2026 20:15:04] INFO [smart_manager.replication.sender:335] Id: 03d502e0-045e-05a2-3d06-950700080009-14. Sending incremental replica between /mnt2/docker/.snapshots/docker_bazarr_config/docker_bazarr_config_14_replication_1 -- /mnt2/docker/.snapshots/docker_bazarr_config/docker_bazarr_config_14_replication_33
[15/Aug/2026 20:19:07] ERROR [smart_manager.data_collector:1017] Failed to update disk state.. exception: Exception while setting access_token for url(http://127.0.0.1:8000): HTTPConnectionPool(host='127.0.0.1', port=8000): Read timed out. (read timeout=2). content: None
[15/Aug/2026 20:19:09] ERROR [smart_manager.data_collector:1017] Failed to update pool state.. exception: Exception while setting access_token for url(http://127.0.0.1:8000): HTTPConnectionPool(host='127.0.0.1', port=8000): Read timed out. (read timeout=2). content: None
[15/Aug/2026 20:20:04] INFO [smart_manager.replication.sender:335] Id: 03d502e0-045e-05a2-3d06-950700080009-14. Sending incremental replica between /mnt2/docker/.snapshots/docker_bazarr_config/docker_bazarr_config_14_replication_33 -- /mnt2/docker/.snapshots/docker_bazarr_config/docker_bazarr_config_14_replication_34
[15/Aug/2026 20:20:16] ERROR [smart_manager.data_collector:1017] Failed to update disk state.. exception: Exception while setting access_token for url(http://127.0.0.1:8000): HTTPConnectionPool(host='127.0.0.1', port=8000): Read timed out. (read timeout=2). content: None
[15/Aug/2026 20:20:18] ERROR [smart_manager.data_collector:1017] Failed to update pool state.. exception: Exception while setting access_token for url(http://127.0.0.1:8000): HTTPConnectionPool(host='127.0.0.1', port=8000): Read timed out. (read timeout=2). content: None
[15/Aug/2026 20:25:04] ERROR [smart_manager.replication.sender:79] Id: 03d502e0-045e-05a2-3d06-950700080009-14. b'unexpected reply(b\'receiver-error\') for 03d502e0-045e-05a2-3d06-950700080009-14. extended reply: b"b\'Failed to promote the oldest Snapshot to Share.\'. Exception: 500 Server Error: Internal Server Error for url: http://127.0.0.1:8000/api/shares/40/snapshots/docker_bazarr_config_14_replication_1/repclone". Aborting'. Exception: b'unexpected reply(b\'receiver-error\') for 03d502e0-045e-05a2-3d06-950700080009-14. extended reply: b"b\'Failed to promote the oldest Snapshot to Share.\'. Exception: 500 Server Error: Internal Server Error for url: http://127.0.0.1:8000/api/shares/40/snapshots/docker_bazarr_config_14_replication_1/repclone". Aborting'
I dont know how relevant the errors are but I left them in any way.
Receiver
[15/Aug/2026 20:05:22] ERROR [system.osi:264] non-zero code(1) returned by command: ['/usr/sbin/btrfs', 'qgroup', 'destroy', '0/280', '/mnt2/Docker_Main']. output: [''] error: ['ERROR: unable to destroy quota group: Device or resource busy', '']
[15/Aug/2026 20:05:22] ERROR [storageadmin.views.share:393] Error running a command. cmd = /usr/sbin/btrfs qgroup destroy 0/280 /mnt2/Docker_Main. rc = 1. stdout = ['']. stderr = ['ERROR: unable to destroy quota group: Device or resource busy', '']
Traceback (most recent call last):
File "/opt/rockstor/src/rockstor/storageadmin/views/share.py", line 391, in delete
remove_share(share.pool, share.subvol_name, share.pqgroup, force=force)
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/rockstor/src/rockstor/fs/btrfs.py", line 1196, in remove_share
qgroup_destroy(qgroup, root_pool_mnt)
~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/rockstor/src/rockstor/fs/btrfs.py", line 1503, in qgroup_destroy
return run_command([BTRFS, "qgroup", "destroy", qid, mnt_pt], log=True)
File "/opt/rockstor/src/rockstor/system/osi.py", line 266, in run_command
raise CommandException(cmd, out, err, rc)
system.exceptions.CommandException: Error running a command. cmd = /usr/sbin/btrfs qgroup destroy 0/280 /mnt2/Docker_Main. rc = 1. stdout = ['']. stderr = ['ERROR: unable to destroy quota group: Device or resource busy', '']
[15/Aug/2026 20:05:22] ERROR [storageadmin.util:45] Exception: Failed to delete the share (.snapshots/03d502e0-045e-05a2-3d06-950700080009_docker_bazarr_config/docker_bazarr_config_13_replication_1). Error from the OS: Error running a command. cmd = /usr/sbin/btrfs qgroup destroy 0/280 /mnt2/Docker_Main. rc = 1. stdout = ['']. stderr = ['ERROR: unable to destroy quota group: Device or resource busy', '']
Traceback (most recent call last):
File "/opt/rockstor/src/rockstor/storageadmin/views/share.py", line 391, in delete
remove_share(share.pool, share.subvol_name, share.pqgroup, force=force)
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/rockstor/src/rockstor/fs/btrfs.py", line 1196, in remove_share
qgroup_destroy(qgroup, root_pool_mnt)
~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/rockstor/src/rockstor/fs/btrfs.py", line 1503, in qgroup_destroy
return run_command([BTRFS, "qgroup", "destroy", qid, mnt_pt], log=True)
File "/opt/rockstor/src/rockstor/system/osi.py", line 266, in run_command
raise CommandException(cmd, out, err, rc)
system.exceptions.CommandException: Error running a command. cmd = /usr/sbin/btrfs qgroup destroy 0/280 /mnt2/Docker_Main. rc = 1. stdout = ['']. stderr = ['ERROR: unable to destroy quota group: Device or resource busy', '']
I split the log here because the start is unrelated to the current replication itself. But this was error thrown when I deleted the share created by the old failed replication task to clean things up. Interestingly the share was still deleted correctly.
I also did a btrfs quota rescan of my Pool and Share just to be sure.
At 20.25 the fourth replication was sent over and this is when the replication failed on the receiver.
[15/Aug/2026 20:25:02] INFO [storageadmin.views.snapshot:61] Supplanting share (03d502e0-045e-05a2-3d06-950700080009_docker_bazarr_config) with snapshot (.snapshots/03d502e0-045e-05a2-3d06-950700080009_docker_bazarr_config/docker_bazarr_config_14_replication_1).
[15/Aug/2026 20:25:02] ERROR [system.osi:264] non-zero code(1) returned by command: ['/usr/sbin/btrfs', 'qgroup', 'destroy', '0/283', '/mnt2/Docker_Main']. output: [''] error: ['ERROR: unable to destroy quota group: Device or resource busy', '']
[15/Aug/2026 20:25:02] ERROR [storageadmin.util:45] Exception: Error running a command. cmd = /usr/sbin/btrfs qgroup destroy 0/283 /mnt2/Docker_Main. rc = 1. stdout = ['']. stderr = ['ERROR: unable to destroy quota group: Device or resource busy', '']
Traceback (most recent call last):
File "/opt/rockstor/src/rockstor/storageadmin/views/clone_helpers.py", line 92, in create_repclone
remove_share(share.pool, share.name, PQGROUP_DEFAULT)
~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/rockstor/src/rockstor/fs/btrfs.py", line 1196, in remove_share
qgroup_destroy(qgroup, root_pool_mnt)
~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/rockstor/src/rockstor/fs/btrfs.py", line 1503, in qgroup_destroy
return run_command([BTRFS, "qgroup", "destroy", qid, mnt_pt], log=True)
File "/opt/rockstor/src/rockstor/system/osi.py", line 266, in run_command
raise CommandException(cmd, out, err, rc)
system.exceptions.CommandException: Error running a command. cmd = /usr/sbin/btrfs qgroup destroy 0/283 /mnt2/Docker_Main. rc = 1. stdout = ['']. stderr = ['ERROR: unable to destroy quota group: Device or resource busy', '']
[15/Aug/2026 20:25:02] ERROR [smart_manager.replication.receiver:100] b'Failed to promote the oldest Snapshot to Share.'. Exception: 500 Server Error: Internal Server Error for url: http://127.0.0.1:8000/api/shares/40/snapshots/docker_bazarr_config_14_replication_1/repclone
Edit: The log of the receiver is complete as is, I only split it for convenience.