While attempting to rsync some media to my 4x 4TB RAID 5 setup on Rockstor I’ve been running into problems of it hanging. The first time I set it to run at night and woke up to transfer on the client frozen, and the server unresponsive even from keyboard.
I rebooted and tried again. It loaded Rockstor then froze again (output an unrelated video kernel message then that was it).
I rebooted and tried again. It loaded. I commenced the transfer again, and this time it failed after a while. The filesystem was forced into Read Only by BTRFS, and the gui is not loading properly.
Error Traceback provided from dmesg:
[ +0.000002] blk_update_request: I/O error, dev sdc, sector 161872896
[ +0.002875] program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO
[ +0.000060] program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO
[ +0.000029] program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO
[ +0.000165] ata5: EH complete
[ +0.008366] BTRFS error (device sdc): bad tree block start 465873467851120433 722485248
[ +0.000667] program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO
[ +0.013292] BTRFS error (device sdc): bdev /dev/sdf errs: wr 0, rd 2, flush 0, corrupt 0, gen 0
[ +0.000165] program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO
[ +0.001507] program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO
[ +0.000052] program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO
[ +0.000046] program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO
[ +0.000069] program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO
[ +0.000313] program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO
[ +0.000105] program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO
[ +0.000073] program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO
[ +0.028422] BTRFS error (device sdc): bad tree block start 5942428209101057412 722485248
[ +0.010169] program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO
[ +0.000083] program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO
[ +0.000187] program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO
[ +0.000035] program smartctl is using a deprecated SCSI ioctl, please convert it to SG_IO
[ +0.000572] ------------[ cut here ]------------
[ +0.000054] WARNING: CPU: 3 PID: 3894 at fs/btrfs/extent-tree.c:2946 btrfs_run_delayed_refs+0x292/0x2d0 [btrfs]
[ +0.000002] BTRFS: Transaction aborted (error -5)
[ +0.000062] Modules linked in: xt_conntrack ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype iptable_filter nf_nat nf_conntrack ext4 jbd2 mbcache amd_freq_sensitivity kvm_amd kv
[ +0.000005] CPU: 3 PID: 3894 Comm: btrfs-transacti Not tainted 4.6.0-1.el7.elrepo.x86_64 #1
[ +0.000002] Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./QC5000-ITX, BIOS P1.10 09/09/2014
[ +0.000006] 0000000000000286 000000009e0e2774 ffff8801fb5fbd08 ffffffff81344adf
[ +0.000003] ffff8801fb5fbd58 0000000000000000 ffff8801fb5fbd48 ffffffff8107fb91
[ +0.000003] 00000b82ced19950 ffff8800d66f9140 ffff8800ced19ac0 ffff8802122a6800
[ +0.000001] Call Trace:
[ +0.000011] [<ffffffff81344adf>] dump_stack+0x63/0x84
[ +0.000006] [<ffffffff8107fb91>] __warn+0xd1/0xf0
[ +0.000004] [<ffffffff8107fc0f>] warn_slowpath_fmt+0x5f/0x80
[ +0.000026] [<ffffffffa036d042>] btrfs_run_delayed_refs+0x292/0x2d0 [btrfs]
[ +0.000006] [<ffffffff810ed768>] ? del_timer_sync+0x48/0x50
[ +0.000028] [<ffffffffa0382b56>] btrfs_commit_transaction+0x56/0xa70 [btrfs]
[ +0.000029] [<ffffffffa037db92>] transaction_kthread+0x1d2/0x200 [btrfs]
[ +0.000027] [<ffffffffa037d9c0>] ? btrfs_cleanup_transaction+0x580/0x580 [btrfs]
[ +0.000005] [<ffffffff8109f738>] kthread+0xd8/0xf0
[ +0.000006] [<ffffffff81716642>] ret_from_fork+0x22/0x40
[ +0.000004] [<ffffffff8109f660>] ? kthread_park+0x60/0x60
[ +0.000002] ---[ end trace b6b71e109ba84b8f ]---
[ +0.000006] BTRFS: error (device sdc) in btrfs_run_delayed_refs:2946: errno=-5 IO failure
[ +0.000002] BTRFS info (device sdc): forced readonly
The earlier errors repeat a few time for each device. I can send the full dmesg log.
I ran memtest through 4 passes before the install.