Failed to import pool

McFaul · August 1, 2016, 9:00am

Ther eis another thread with the same title which was fixed, but it seems to have been a code change, there are no steps i can see to fix this

Brief description of the problem

I am unable to import an existing pool

Web-UI screenshot

Houston, we’ve had a problem.

Failed to import any pool on this device(ata-WDC_WD80EFZX-68UW8N0_VKGUTXKX). Error: deadlock detected DETAIL: Process 5316 waits for ShareLock on transaction 56236; blocked by process 5351. Process 5351 waits for ShareLock on transaction 56235; blocked by process 5316. HINT: See server log for query details.

Error Traceback provided on the Web-UI


            Traceback (most recent call last):
  File "/opt/rockstor/src/rockstor/storageadmin/views/disk.py", line 379, in _btrfs_disk_import
    do.save()
  File "/opt/rockstor/eggs/Django-1.6.11-py2.7.egg/django/db/models/base.py", line 545, in save
    force_update=force_update, update_fields=update_fields)
  File "/opt/rockstor/eggs/Django-1.6.11-py2.7.egg/django/db/models/base.py", line 573, in save_base
    updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
  File "/opt/rockstor/eggs/Django-1.6.11-py2.7.egg/django/db/models/base.py", line 635, in _save_table
    forced_update)
  File "/opt/rockstor/eggs/Django-1.6.11-py2.7.egg/django/db/models/base.py", line 679, in _do_update
    return filtered._update(values) > 0
  File "/opt/rockstor/eggs/Django-1.6.11-py2.7.egg/django/db/models/query.py", line 510, in _update
    return query.get_compiler(self.db).execute_sql(None)
  File "/opt/rockstor/eggs/Django-1.6.11-py2.7.egg/django/db/models/sql/compiler.py", line 980, in execute_sql
    cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
  File "/opt/rockstor/eggs/Django-1.6.11-py2.7.egg/django/db/models/sql/compiler.py", line 786, in execute_sql
    cursor.execute(sql, params)
  File "/opt/rockstor/eggs/Django-1.6.11-py2.7.egg/django/db/backends/util.py", line 53, in execute
    return self.cursor.execute(sql, params)
  File "/opt/rockstor/eggs/Django-1.6.11-py2.7.egg/django/db/utils.py", line 99, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/opt/rockstor/eggs/Django-1.6.11-py2.7.egg/django/db/backends/util.py", line 53, in execute
    return self.cursor.execute(sql, params)
OperationalError: deadlock detected
DETAIL:  Process 5316 waits for ShareLock on transaction 56236; blocked by process 5351.
Process 5351 waits for ShareLock on transaction 56235; blocked by process 5316.
HINT:  See server log for query details.

phillxnet · August 5, 2016, 10:32am

@McFaul In reply to your post in the “Dev log for 3.8-15”

Answering here so as not to clutter the dev log post.

Sorry to not have replied earlier it’s just that I don’t have much to offer on your reported error. It looks like a simple database deadlock, except that these types of errors are not that often ‘simple’. As you say it could be related however there have been no other reports of this (as yet) and from the tests I’ve done import works exactly as was. The change to by-id was deep but didn’t really change much structurally.

I think this is a case of more info may help ie what is the nature of your pool and if you could paste the output of the following command here it may help others identify what the problem is.

btrfs fi show

Also a screen grab of your Disk page may help as well.

When I have seen db contention errors I have often simply tried again and had success as they are often very time dependant / transient. It very much depends what’s going on in the background when you attempt the pool import. I take it you have tried the import more than once?

Sorry but db contention is a little out of my area currently, but the slight change in pattern of db access by the by-id name change may have triggered something but quite a few imports have been done since this change and this db issue hasn’t arisen.

Another possible cause is if you have your Disks page on an auto refresh when you attempt the pool import that may increase the likely hood of this contention, due to the additional db pooling. It shouldn’t of course but there may be a bug where db actions are not atomic where they need to be or the like.

Also note that all your devices must not have serial issues as this may also cause problems.

McFaul · August 5, 2016, 1:19pm

Hi,

having done a little more digging, it turns out that my issue is slightly more involved than i suspected. that pool cannot be mounted manually. it seems i have quite an issue with the underlying file system.

I recently removed one disk from the pool (as in BTRFS device delete, which completed successfully after several days) then it wouldnt boot…

`[root@rockstor mnt2]# mount /dev/sds /mnt2/1
mount: wrong fs type, bad option, bad superblock on /dev/sds,
       missing codepage or helper program, or other error

       In some cases useful info is found in syslog - try
       dmesg | tail or so.`

even mount -ro,recovery wont do it

so i tried

[root@rockstor mnt2]# btrfs check /dev/sds checksum verify failed on 78555517419520 found 1974988D wanted EC623494 checksum verify failed on 78555517419520 found 1974988D wanted EC623494 checksum verify failed on 78555517419520 found 29A91167 wanted 8B70494D checksum verify failed on 78555517419520 found 1974988D wanted EC623494 bytenr mismatch, want=78555517419520, have=12198095107859028614 checksum verify failed on 78555492007936 found F8E8312F wanted 0FB3BE53 checksum verify failed on 78555492007936 found F8E8312F wanted 0FB3BE53 checksum verify failed on 78555492007936 found 32C0F59B wanted 1CF8B7B6 checksum verify failed on 78555492007936 found DC8C2802 wanted 9C5466C0 checksum verify failed on 78555492007936 found F8E8312F wanted 0FB3BE53 bytenr mismatch, want=78555492007936, have=13330466552269131069 Couldn't open file system

[root@rockstor mnt2]# btrfs restore -D /dev/sds /mnt2/2/Movies4k
checksum verify failed on 78555517419520 found 1974988D wanted EC623494
checksum verify failed on 78555517419520 found 1974988D wanted EC623494
checksum verify failed on 78555517419520 found 29A91167 wanted 8B70494D
checksum verify failed on 78555517419520 found 1974988D wanted EC623494
bytenr mismatch, want=78555517419520, have=12198095107859028614
checksum verify failed on 78555492007936 found F8E8312F wanted 0FB3BE53
checksum verify failed on 78555492007936 found F8E8312F wanted 0FB3BE53
checksum verify failed on 78555492007936 found 32C0F59B wanted 1CF8B7B6
checksum verify failed on 78555492007936 found DC8C2802 wanted 9C5466C0
checksum verify failed on 78555492007936 found F8E8312F wanted 0FB3BE53
bytenr mismatch, want=78555492007936, have=13330466552269131069
Could not open root, trying backup super
warning, device 7 is missing
warning, device 6 is missing
warning, device 5 is missing
warning, device 12 is missing
warning, device 11 is missing
warning, device 9 is missing
warning, device 4 is missing
warning, device 3 is missing
warning, device 2 is missing
warning, device 8 is missing
bytenr mismatch, want=78556015362048, have=0
ERROR: cannot read chunk root
Could not open root, trying backup super
warning, device 7 is missing
warning, device 6 is missing
warning, device 5 is missing
warning, device 12 is missing
warning, device 11 is missing
warning, device 9 is missing
warning, device 4 is missing
warning, device 3 is missing
warning, device 2 is missing
warning, device 8 is missing
bytenr mismatch, want=78556015362048, have=0
ERROR: cannot read chunk root
Could not open root, trying backup super`

finally

[root@rockstor mnt2]# btrfs-find-root /dev/sds
Superblock thinks the generation is 56073
Superblock thinks the level is 1
Found tree root at 72701233987584 gen 56073 level 1
Well block 72701214523392(gen: 56072 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 70632449130496(gen: 56071 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 70632100249600(gen: 56070 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 69587596034048(gen: 56069 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 69587056312320(gen: 56068 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 67526344032256(gen: 56067 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 66472610447360(gen: 56066 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 66472423751680(gen: 56065 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 66471797604352(gen: 56064 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 65407859818496(gen: 56063 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 65407547310080(gen: 56062 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 64392209219584(gen: 56061 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 63367027965952(gen: 56060 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 63366561447936(gen: 56059 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 60126110433280(gen: 56058 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 59187274252288(gen: 56057 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 58123094425600(gen: 56056 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 58122762371072(gen: 56055 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 56265840672768(gen: 56054 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 53131948998656(gen: 56053 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 51128654200832(gen: 56052 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 51128229511168(gen: 56051 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 49320826273792(gen: 56050 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 49320269316096(gen: 56049 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 48430579695616(gen: 56048 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 48430173962240(gen: 56047 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 48429748862976(gen: 56046 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 48429601587200(gen: 56045 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 48429674348544(gen: 56044 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 48429674102784(gen: 56044 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 48429534117888(gen: 56044 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 48429534085120(gen: 56044 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 48429534019584(gen: 56044 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 48429533921280(gen: 56044 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 48429533315072(gen: 56044 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 48429533298688(gen: 56044 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 48429532463104(gen: 56044 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 48429532446720(gen: 56044 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 48429532430336(gen: 56044 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 48429532413952(gen: 56044 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 48429532397568(gen: 56044 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 48429532315648(gen: 56044 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 48429532299264(gen: 56044 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 48429517979648(gen: 56044 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 48429603749888(gen: 56043 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 48429602160640(gen: 56043 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 48429602062336(gen: 56043 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 48429535789056(gen: 56042 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 48429535756288(gen: 56042 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 48429534560256(gen: 56042 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 48429534380032(gen: 56042 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 48429534363648(gen: 56042 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 48429533675520(gen: 56042 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 48429533216768(gen: 56042 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 48429533184000(gen: 56042 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 48429532971008(gen: 56042 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 48429532872704(gen: 56042 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 46863803793408(gen: 56041 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 46863596978176(gen: 56040 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 46863401828352(gen: 56039 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 46863337603072(gen: 56038 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 46863232860160(gen: 56037 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 46863223046144(gen: 56036 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 46863213363200(gen: 56036 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 46863213330432(gen: 56036 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 46862992850944(gen: 56035 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 46862892285952(gen: 56034 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 45886577016832(gen: 56033 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 45886126997504(gen: 56032 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 45885944348672(gen: 56031 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 45885738893312(gen: 56030 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 45015631708160(gen: 56029 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 45015461888000(gen: 56028 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 45015279960064(gen: 56027 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 45015271227392(gen: 56026 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 45015185309696(gen: 56025 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 43892692156416(gen: 56024 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 41686927982592(gen: 56023 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 59186640781312(gen: 55971 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 59186596331520(gen: 55971 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 59186594611200(gen: 55971 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 52154053394432(gen: 55965 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 52153667190784(gen: 55964 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 52153719848960(gen: 55963 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 52153719832576(gen: 55963 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 52153719521280(gen: 55963 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 52153718046720(gen: 55963 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 52153717719040(gen: 55963 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 52153717358592(gen: 55963 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 52153716195328(gen: 55963 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 52153715818496(gen: 55963 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 52153714966528(gen: 55963 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 52153714606080(gen: 55963 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 52153714556928(gen: 55963 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 52153711345664(gen: 55963 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 52153711001600(gen: 55963 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 52153685655552(gen: 55963 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 52153683329024(gen: 55963 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 52153683197952(gen: 55963 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 52153683181568(gen: 55963 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 52153683001344(gen: 55963 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 52153682575360(gen: 55963 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 52153682313216(gen: 55963 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 52153676972032(gen: 55963 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 52153674039296(gen: 55963 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 52153672138752(gen: 55963 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 52153666699264(gen: 55963 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 52153665978368(gen: 55963 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 42799823290368(gen: 55947 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 42799543566336(gen: 55946 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 42799543517184(gen: 55946 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 42799543468032(gen: 55946 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 42799543222272(gen: 55946 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 42799543025664(gen: 55946 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 42799542943744(gen: 55946 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 42799542321152(gen: 55946 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 42799542304768(gen: 55946 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 42799541846016(gen: 55946 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 42799539765248(gen: 55946 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 42799538798592(gen: 55946 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 42799533146112(gen: 55946 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 42799504654336(gen: 55946 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 42799491579904(gen: 55946 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 53131055906816(gen: 55832 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 51128112988160(gen: 55558 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 45014617112576(gen: 55319 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 72701044490240(gen: 53934 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 72701044441088(gen: 53934 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 72701044408320(gen: 53934 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 72701044178944(gen: 53934 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 69586927992832(gen: 53932 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 46862784937984(gen: 53905 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 46862788116480(gen: 53903 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 70632984363008(gen: 49567 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 70632955592704(gen: 49567 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 70632951431168(gen: 49567 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 70632603648000(gen: 49566 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 70632563179520(gen: 49563 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 70632561819648(gen: 49563 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 70632449015808(gen: 49562 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 70632447967232(gen: 49562 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 70632416067584(gen: 49558 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 70632415543296(gen: 49558 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 70632415510528(gen: 49558 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 70632286306304(gen: 49557 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 70632088518656(gen: 49552 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 70632079212544(gen: 49551 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 70632079196160(gen: 49551 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 70632079179776(gen: 49551 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 70632078163968(gen: 49551 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 70632078131200(gen: 49551 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 70632073248768(gen: 49547 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 70632073232384(gen: 49547 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 70631881408512(gen: 49546 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 70631874920448(gen: 49546 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 70631874887680(gen: 49546 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 69588001505280(gen: 49541 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 69587866828800(gen: 49540 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 69587866304512(gen: 49540 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 69587860881408(gen: 49538 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 69587860865024(gen: 49537 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 69587860848640(gen: 49537 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 69587696599040(gen: 49535 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 69587696582656(gen: 49535 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 69587696549888(gen: 49535 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 69587457032192(gen: 49532 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 69587457015808(gen: 49532 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 69587453722624(gen: 49532 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 69587453673472(gen: 49532 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 69587388317696(gen: 49531 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 69587387252736(gen: 49531 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 69587341836288(gen: 49530 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 69587341803520(gen: 49530 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 69587340771328(gen: 49530 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 69587340754944(gen: 49530 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 69587291471872(gen: 49530 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 69587272564736(gen: 49527 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 69587249414144(gen: 49526 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 68571784200192(gen: 49524 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 68571718795264(gen: 49522 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 68571718221824(gen: 49522 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 68571571666944(gen: 49521 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 68571571650560(gen: 49521 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 68571547287552(gen: 49521 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 68571547238400(gen: 49521 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 68571494301696(gen: 49520 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 68571493269504(gen: 49520 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 68571493793792(gen: 49517 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 68571493777408(gen: 49517 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 68571453833216(gen: 49516 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 68571453816832(gen: 49516 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 68571449393152(gen: 49516 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 67527342997504(gen: 49515 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 67527342964736(gen: 49515 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 67526278905856(gen: 49514 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 66472781316096(gen: 49511 level: 1) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 66472597553152(gen: 49509 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 66472579678208(gen: 49506 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 66472472690688(gen: 49506 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 66472442347520(gen: 49506 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 65407762300928(gen: 49505 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 62351081127936(gen: 49504 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 62350934196224(gen: 49504 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 62350234533888(gen: 49503 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 62350612398080(gen: 49502 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 60125887217664(gen: 49502 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 60125461659648(gen: 49501 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 60125297868800(gen: 49501 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 60125279977472(gen: 49501 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 58122582671360(gen: 49500 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 58122582654976(gen: 49500 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 58122582638592(gen: 49500 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 58122582622208(gen: 49500 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 58122513399808(gen: 49500 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1
Well block 58122506354688(gen: 49500 level: 0) seems good, but generation/level doesn't match, want gen: 56073 level: 1

1

Any thoughts?