Is there a technical reason (centOS installer doesn’t support?) that Rockstor doesn’t have an option to install to a full disk btrfs with subvols? Doing that would allow an easy way to add redundancy to the boot drive and snapshots to roll back on a bad update or the like.
The only issue I can think of is that swap would have to be mounted via a loop device and take a performance hit but to me it seems worth the advantages. ArchLinux at least even has an official script for mounting loop as swap.
I’m hopefully going to try and give this a shot later this week and report back.
The problem doesn’t seem to be with rockstor but rather with centOS or at least the installer. It refuses to install without a separate ext4 /boot partition which kinda defeats the purpose. It also won’t let you create and mount subvolumes from the CLI for the install partitioning. Btrfs has to be done by the installer and isn’t quite clear if it is making subvols or no. When I set up btrfs manually I put root as a separate subvol for snapshot purposes and the installer told me root can’t be a subvol for some reason.
I think that if one could install the system manually via bootstrap most likely it would work fine. Problem is I can’t find much info on that for rpm distros to try it. The most recent info I found was for CentOS 6.5 and Fedora 18 and I’m not familiar enough with them to know what packages are needed.
Alright let you know the results of further experiments.
Alright it works! but it is a bit of a pain though easier that what i was thinking in my last post (should have thought of this to start).
The problem is the installer it absolutely will not install without a separate /boot partition. So what I did was install the system with a separate /boot and made a subvol for it. then from a live cd I chrooted into my new system and deleted /boot and then installed the newest version of grub and re ran grub2-mkconfig and volia it works (I added the btrfs mckiptio hooks too but IDK if that is necessary) doesn’t seem to be any issues so far and adding a disk then re-balancing to RAID 1 didn’t give any errors.
Swap as a loop device seems not to have caused any real performance issues storage wise though I still need to try it with some rock-ons.
This is great @Spectre694! Care to write a howto post with detailed steps? I think it’s going to be very useful for everybody. I am open to any of these options
A wiki post right here on the forum. For this just create a new post and I’ll convert it into a wikified post.
A howto in our docs via pull request on the docs repo
A blog post in our blog with same info.
I know it can be time consuming to do any of this, but I hope you can do (1) at the very least and we can take it from. If you are interest, a doc pull request would be welcomed and I’d also be happy to have you write a guest blog post on our blog. Let me know.
Definitely wouldn’t mind writing up a guide or something but I believe there is still testing and some stuff to sort out before doing so. Nothing too bad just need to find a little time first.
I think the chroot step isn’t necessary and this can probably be done from the installed system which would simplify things (probably just typed something wrong the first time I’m going to try again no reason I can think of it wouldn’t work)
2)swap has to be manually turned on after boot (a simple script or maybe systemd service? have to look into that)
3)auto mount of degraded array’s on boot. As it stands it doesn’t unless you use the degraded option on every mount. not sure what that does to a good array though.(degraded can be added to fstab and GRUB cmd line and it works though I doubt that is the best way to do it)
grub mkconfig and a balance would probably half to be run on every update (kernel specifically) but I’m not sure that can really be avoided
1 and 2 should be easy enough to verify/fix. 3 needs more research and opinions on if this is necessary/wanted and 4 IDK that anything can be done there.
@phillxnet Sorry for the late post haven’t had too much time to play with this recently.
Good find that actually answered why I haven’t been able to get anaconda to install to certain subvol setups. plus there was some other goodies in there too.
I think I will play with the usb /boot idea even though that makes snapshotting it a little harder. It might just be easier than fighting with grubby and anaconda all the time.
Sorry guys but for now this is something that I think we will have to wait on unfortunately. The reason is that this setup is really prone to breakage.
Btrfs RAID on the rootvol work sometimes (rarely apparently I got lucky that first time) but randomly breaks for no reason and refuses to boot again and other times never boots after install. I have yet to figure out what is going on here it has honestly been driving me nuts.
Snapshots work but they did before too. Still not as nice as I’d like though due to anaconda’s insistence on putting btrfs on a full disk partition and making everything top level subvols. There is also the issue of /boot not rolling back with the system that breaks things about half the time.
I would say lvm might help but no you can’t put boot on lvm either -_-
I’m going to get some info together for some bug reports but based on the track record of this long standing bug red hat actually has no interest in getting btrfs support fixed.
On the plus side I found a bunch of disk layouts that anaconda will happily install to that just don’t work either.
That said it is still easy enough to actually move /boot to a separate subvol so long as you remember to update GRUB every time you update the kernel. Snapshots work like this but RAID does not.
You mean like a patch that makes grubby behave correctly? Ya there was one but at this point it is several years old and it was kinda messy too start with. Grubby is apparently very inflexible when it comes to new/different ways to boot which I find somewhat ironic for something that was designed to make red hat more flexible in terms of bootloaders it can use and platforms it could be installed to.
The problem still exists as of Fedora 24 (still rawhide I believe) and with no real progress having been made. The specific bug is currently sitting with no priority attached.
Anyway nothing has really changed since last I’ve posted. It is still possible to do this it just isn’t practical to put /boot on the btrfs mirror. It is a pain to do and breaks constantly. The rest of the disk can be btrfs just not /boot. Which for me kinda defeats the purpose of what I was trying to do.