OK. I found some guides online and found that in the line
./configure --prefix=/usr/sbin
None og the other guides seemed to use the --prefix=/usr/sbin, so I tried again without it.
And voila, after a reboot my btrfs-progs now reads:
btrfs version
btrfs-progs v4.19.1
I hope this didn’t break anything 
The reason I did that (with the prefix) is because in the Rockstor code (at least the version I have) the path to btrfs is hardcoded in a few places:
/opt/rockstor/src/rockstor/scripts/qgroup_clean.py:BTRFS = ‘/usr/sbin/btrfs’
/opt/rockstor/src/rockstor/scripts/qgroup_maxout_limit.py:BTRFS = ‘/usr/sbin/btrfs’
/opt/rockstor/src/rockstor/scripts/qgroup_test.py:BTRFS = ‘/usr/sbin/btrfs’
So if you install btrfs-progs from source, and let it install into /usr/local/bin, then Rockstor will continue to use the old version in /usr/sbin (which is v4.12) and not the new one (which is v4.19).
So by using your method, Rockstor will use the new version, but via SSH you’ll get the old version?
This seems rather counter intuitive
Or why did it show 4.12 via SSH?
Just trying to understand
And by doing it two times in different ways, I have two copies of 4.19.1 installed, being used by different parts of RockStor?
When logged in via SSH, it’ll use whatever is on your path. So you must have two versions of the btrfs-profs installed.
You can check the location that is being used in your ash session by typing:
which btrfs
On my system I only have the tools in both /use/sbin and /usr/local/bin
@kupan787 and @KarstenV
Ben, I used the latest instructions that you so graciously posted, and I ran into the same issue as Karsten. After using the same “fix” the tools version also changed to btrfs-progs v4.20.1
“which brtfs” is now giving back: /usr/local/bin/btrfs,
before which brtfs showed the /usr/sbin/btrfs (and when I now directly query the version in that directory it still gives the version btrfs-progs v4.12.).
And the Rockstor version I currently play with (3.9.2-47) has the same hard-coded paths …
Just to give a short update - I’ve been running with the updated pieces for the last 40 days and have not run into any issues, though, I have to admit I am not doing anything radical with the Rockstor other than file storage and a few Rockons. We will see what happens, on the next update …
Have anybody tried the newer 5.1 kernel with the 5.1(.1) btrfs progs?
Any easy installation instructions?
Just replace 4.20 with 5.10 in the above commands?
Yes, I updated to the 5.1.15 kernel and btrfs-progs v5.1.1. Just replace version numbers.
working with 5.4, just update to python36-devel
Just to add to one of @KarstenV’s post earlier, I still had trouble to install the btrfs-progs into the “correct” directory (i.e. so the hard-coded paths mentioned above would point to the latest version. I changed the command lines from this:
to this:
./autogen.sh
./configure --prefix=/usr --bindir=/sbin --libdir=/lib
make
make install
Now the version for the RockStor referenced programs is:
/usr/sbin/btrfs --version
btrfs-progs v5.4
Just another update, it appears that the latest btrfs-progs require a couple more dependencies, so here’s my full steps for successful upgrade to kernel 5.7.2-1.el7.elrepo.x86_64 and btrfs-progs v5.6.1:
yum -y group install "Development Tools" # Not sure if this is needed, but can't hurt
rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org # Import the key for the mainline kernel repo
yum -y install https://www.elrepo.org/elrepo-release-7.el7.elrepo.noarch.rpm # Install the elrepo repo file
yum -y --enablerepo=elrepo-kernel update # elrepo-kernel is deactivated by default, enable it and update the system
yum list installed | grep kernel # You should see multiple kernel versions now
kernelver=$(grubby --default-kernel | sed 's@/boot/vmlinuz-@@') # The default kernel will now be the latest
sed -i "/SUPPORTED_KERNEL_VERSION/c\SUPPORTED_KERNEL_VERSION = '${kernelver}'" /opt/rockstor/src/rockstor/settings.py # Update the supported kernel version
grep SUPPORTED /opt/rockstor/src/rockstor/settings.py # Verify the setting is changed correctly
reboot # To ensure that the build happens with the new kernel
uname -r # After reboot to be sure we're running the new kernel
# Install required devel packages for building btrfs-tools
yum -y install asciidoc e2fsprogs-devel libacl-devel libattr-devel libblkid-devel libuuid-devel libzstd-devel lzo-devel python36-devel xmlto zlib-devel
mkdir ~/src; cd ~/src # Make a new source directory to hold our btrfs-progs source
git clone git://git.kernel.org/pub/scm/linux/kernel/git/kdave/btrfs-progs.git # Grab the latest source, cd into the dir and git pull if you have it already
cd btrfs-progs # Change to the source dir
./autogen.sh # Begin the configuration
./configure --prefix=/usr --bindir=/sbin --libdir=/lib # Configure the build
make # Compile the source
yum remove btrfs-progs # Remove the repo version of btrfs-progs to avoid update conflicts
make install # Install the source
btrfs --version # Verify
man btrfs # Double verify@StephenBrown2
that works well on 3.9
i got it to work on 3.9 (i don’t recommend running “yum remove btrfs-progs” as that deletes rockstor dependency witch well removes rockstor as i found) maybe better to just turn off auto update
can’t seem to get it to default to 5.9 it keeps on picking the last version
,
on rocstor 4.0 as it uses opensuse i have managed to get it to update the kernel 5.9 but i can’t do the btrfs compile because it needs python-3.6 i get “Package requirements (python-3.6) were not met” can’t seem to work out how to get it to recognize its installed (python3-3.6.10-lp152.4.6.2.x86_64) says its already installed
i am new on here and interested in the project (be nice if tumbleweed would build with rockstore but missing rockstor 4.0.4 rpm file)
@lexx Welcome to the Rockstor community.
Yes we did build for Tumbleweed for a while but our various technical debt issues ended up making that impossible as we are still Python2 and have some other old dependencies that were also similarly dropped from Tumbleweed so we could no longer build the rpms except on systems that already had these dependencies grand fathered in. The last rpm we released for Tumbleweed was 4.0.1-0 but again it will only install on a Tumbleweed that already has these now missing dependencies pre-installed. And the rockstor-installer recipe fails as as these dependencies are now no longer available in the default Tumbleweed. But for a good few months we had a fully working Tumbleweed and it help us in seeing what was on the way. Maybe once we have our various technical dept issues resolved we can again return to building for Tumbleweed.
We do hope in a month or two to start releasing rpms for Leap 15.3 so that would be another avenue for newer kernels/btrfs stack. But do bear in mind that the openSUSE kernels and btrfs-progs get many backports form the SuSE/openSUSE team so they are often more up-to-date that their main number would indicate, at least in the btrfs sense. Examining the openSUSE changelog can help to see what has been backported. And given SuSE/openSUSE are very active in btrfs development they are ideally placed to do this back-porting for their own kernels.
Hope that helps and thanks for sharing your Rockstor experiences.
i don’t believe they backport any btrfs features in older kernels and btrfs progs in opensuse (as in a clean install of opensus 15.2 and updated i could not use raid1c3 profile)
hopefully they will be using a newer kernel with 15.3 (but that is like july 2021) , as really a should be using raid1c3 metadata with raid6
raid 5 gui profile should be using raid1 for metadata on rockstor
as a short term stop gap on rockstor can you make RAID6 use a RAID1 metadata with a forced metadata balance after a disk goes missing on raid6 profile, if the 2 disks that go puff that has the metadata you don’t have a filesystem anymore, the force metadata balance is to make sure it shuffles the metadata onto another working disk so it restores the dual metadata redundancy (i would not do a full auto balance just in case there isn’t enough space on the remaining disks for data)
or just update to 5.4 or 5.9 opensuse stable kernel as it seems to be working fine for me on rockstor as in my other post (significantly easier to do in with 4.0.4 due to using opensuse, just 6 commands and a reboot, on 3.9 had to install a bunch of stuff to get it to install kernel and btrfs progs)
@lexx Re:
Currently we don’t recommend the parity raids for production at all. And this stance stands for the time being until, as you say, the Raid1c2/3 are in the default kernel. We wont be installing non default kernels either unfortunately as we need the stability of the ‘distro’ default kernel as then folks know where they are. We made this same mistake and failed in it’s maintenance with our CentOS offering. We just don’t have the developer time currently to have this. But as you have shared, it’s not all that bad to do for ones self.
This may be true but they do backport fixes. I believe @Flox posted a while back on grepping the kernel openSUSE kernel changelog for proof of backports.
Totaly agree that getting raid1c2/3 in would be great. But first we have to surface what’s actually in-play then move to picking apart our current hardwired state re tying metadata and data raid levels together. Except in the case of single where we override metadata to become dup:
I’m personally looking forward to having greater dexterity within the Web-UI re raid levels but all in good time. We have some more pressing issues to attent to first such as our broken AD in Rockstor 4. Plus we have a mountain of technical debt re Python 2 and old Django. So in short the plan is to first get out current ‘Built on openSUSE’ offering to be at feature parity with our prior becoming-legacy CentOS offering. Then we move to addressing the technical debt. However on the raid levels we would welcome pull requests that surface the data and metadata raid levels first. And given we do, at the lowest levels have these as seperate entities I think this would be the first step. Then we ensure there are adequate fences to inform folks of the additional dead ends re raid level conversions. Take a look at:
for the current number of related user messages re our currently supported btrfs raid levels. These would all have to be re-assessed and well tested once we move to seperate raid levels for data and metadata, and would grow still further once we add the raid1c* variants. All doable but we are an appliance and as such folks depend on us to be relatively sane. This is a tricky thing to pull off with btrfs extreme flexibility.
Thanks again for sharing your findings/thoughts. All good stuff but easier said than done and we have to very soon approach our now old technology. We are are least on the verge of the Rocksor 4 and that is a major relieve as the CentOS variant with it’s anchinet kernel was just not what we wanted. So at least we are now in the position of:
But that took a couple of years of available developer time and in the process we have address some technical debt, but not all. It’s all quite frustrating really but at the same time we have also adopted the Arm64 arch so it’s all comming together bit by bit but as always it takes longer that anyone involved hopes for. And as you say we have the products of the Jump project just around the corner with Leap 15.3 so I’m really hoping we are not caught out again, as per our earlier Tumbleweed efforts, with our current technical debt. But it will catch up with us eventually and we just dodged a CentOS dead end with our openSUSE rebase as 3.9.2-57 was the last time we were able to build CentOS rpms. In part due to other elements of our build tec debt.
Hope that helps and do take a look at the code. You can change any of the Python code on your own instance and see the results for yourself post a reboot (or rockstor services stop/start).
@phillxnet (not sure if I need to do @ you if I am pressing reply on you)
i like that it saves drafts once you stop typing (this is the cleanest forum i have ever been on)
The only tweak i would do is when a disk is been added to a existing pool it should not be passing any optional commands when adding a disk, as add disk doesn’t need “balance mconvert or/and dconvert” to be used witch it must be doing when a disk is been added to an existing pool vi GUI (i haven’t looked at the code yet as that’s a little new for me to do but got good idea)
As when I use RAID6 for data and RAID1c3 for metadata it’s working fine in rockstor as far as I can tell its only when I add a disk to a pool it must be passing balance mconvert=raid6 and I guess dconvert=raid6 when it shouldn’t be after a disk added to the pool (even though I am picking No to Change raid level, if i pick yes then it should pass mconvert and dconvert when using balance)
a Manual or scheduled balance doesn’t change the data or metadata layout
Does using metadata as its own RAID1 layer break something as its displayed the same just says RAID1 instead of 6 or 5 (the metadata is already separate from the data, and Single and DUP for metadata shouldn’t be any different from using any raid level for metadata)
I am not fully Linux proficient but got a good idea of how to do Stuff to a point once i have worked out how the command works (the devid:6 option thrown me for a loop when i was trying to expand 5 disks it was only expanding the first devid of 1 , option command is normally = or just space then a number most of the time lol)
I did manage to break the disk page (refresh button was throwing a redbox code error) and the pool page partially when I pulled a disk (disconnected it via virtualbox) and reinserted the same disk first it kept saying missing in pool and it broke the disk page (in the end I had to delete the virtual disk and recreate it so it got a new disk name like “ata-VBOX_HARDDISK_VBc9f77f6a-5b290a63” is that UUid? )
I think I needed to format the disk as it was showing duplicate devids in the pool with 2 different disk Uuid? (at one point the whole btrfs pool broke at boot up until I removed the disk that was making it very unhappy, i ended up just deleting the virtual disk from virtualbox and made a new one)
If you need someone to find bugs quite good at doing that
Last one whenever I go to samba share page witch I have enabled I get lots of popup boxes and I can’t get snapshots to be in a snapshot folder, if I use same syntax for snapshots witch I put the word D in it the folder doesn’t show But if I put any other word in The snapshot share syntax word it shows each individual snapshot in the root dir of the share (like hidden folders ./d+date) I am not at home so going off memory (what syntax do i need to use so snapshots only show in a folder)
And some logs don’t work main one been the kernel log (need clean install to see if it’s the 5.9 kernel that broke that)
ok my samba share page is no longer showing a share when there is one been shown (again 5.9 kernel 5.9) i need to clean 4.0.4 install witch i am going to do now as dont want to be really reporting bugs that i might of caused logs and samba not working due to using 5.9)
i just going to press reply now before i think of somthing else to type
installed 4.0.4 (logs for dmesg still does not work)
i can’t actually work out why it uses “btrfs balance start -f -mconvert=raid5 -dconvert=raid5” when you add a disk to a pool that’s already created unless that code you posted is what at line 1783 is inserting -mconvert=raidX into it resulting it it changing the metadata level when its only a disk add with no been chosen for raid change
as i see it say so it must be running it as stated above even though i picked No on the raid change it changes the raid any way via balance
BTRFS info (device sdd) balance: start -dconvert=raid5 -mconvert=raid5 -sconvert=raid5
if i run a manual balance it uses witch wont change the raid level
BTRFS info (device sdd) balance start -d -m -s
when after a disk has been added to the pool it should only be using “btrfs balance start --full-balance /mount/point”
at a guess when its adding a disk to a pool i think its using the line convert witch has mconvert in it (i am not a coder not don’t have a clue why its passing the mconvert when adding a disk + no raid change option)
if that’s so how would you stop that from happening if no is been chosen for raid level change (if no has been chosen it should not pass anything to start_balance)
this is where it might be doing it passing new raid when raid level is the same level when adding a disk
@lexx Thanks for the feed back. Yes we do have much to do re seperating data and metadata raid levels. But as stated before we first have to surface these levels properly.
The forced balance when adding disks is to re-distribute the data. Agreed it’s not necessary but it’s good to do as otherwise the disk will remain unused until new data is written. We just do a rather overkill full balance all the time irrespective. This is definitely something we should back off from a tad as say we could use filters to to most of the work but not all. But in that case we would have to conditionally apply those filters and that is yet more sophistication that we just don’t have yet. But all in good time.
btrfs does have some existing weaknesses and a temporary disk disconnect and re-connect often then requires a scrub to correct (read re-sync) the pool. The parity raids of 5 and 6 are more fragile in this regard.
Rockstor is just not yet aware that data and metadata can be different. At the lowest level we do keep them seperate but in the higher levels we regard them as the same (except in the case of single as per my last message). All in our pending improvements though. But as stated previously we are still in the final throws of moving over to the openSUSE base and need first to get all our prior functions working there so we can offer a viable upgrade for our existing CentOS variant folks. Once that is achieved (soon as mostly just AD fix) we can return to improving our base function and enhancing across the board. But again we have the non trivial Python2-3 and Django moves to do prior to making any larger changes.
Yes this can happen. Again it’s a maturity thing in btrfs. There is work ongoing upstream to address this and make it easier to release these if need be. We will take advantage of these things as they appear. Disk management / release is a little weak in the btrfs but again upstream improvements are, as always, ongoing.
That one is worth a specific post, and it’s often advisable to do this to help focus a discussion on a particular issue to more rapidly arrive at a conclusion or two.
That would be good, kernel changes can have all sorts of knockons, but I would be surprised in this case. Again a specific thread would be good in you fancy perusing that one. Assuming you get some takers. We can’t really focus on custom (read non distro default) kernel arrangements as we are just not a big enough dev team for that. But good her hear findings as always.
We do force the raid levels currently. That way we asset what is intended. A partial balance or raid level change can leave some data/metadata in different btrfs raid levels so we just (simple mindedly) asset it via a full balance. That code I sighted in where it happens. Again we lack sophistication re different raid levels for data/metadata and we have some way to go to make this so, but not that far actually. But we do have very limited development resources (read humans contributing code). So if you do fancy diving into the code you may be surprised at how readable it actually is. I’d start at the lowest levels and track the exact moment we combine data and metadata. Again, adding reporting of both would help as currently I believe we simply report data raid levels and adding metadata raid levels to the Web-UI would be a nice start. But we end up merging data and metadata btrfs raid levels for simplicity and have as yet not gotten around to addressing them separately. Do consider looking at the code though as it’s definitely something we would like to address going forward. Probably in our next testing run once we have the first official stable our for the ‘Build on openSUSE’ endeavour.
Agreed but we don’t, as that is for predictability, our balance is, as stated, unsofisiticated and we could do with giving it more intelligence. But it can’t be done in isolation as the rest of the system also assumes (in the higher levels anyway) that data and metadata are paired and the same. It was a short cut to getting to the Appliance stage earlier. We can retrofit as we go and this gave less complexity for earlier development is my understanding. But our Django model for the pool does have seperate fields for data and metadata we just need more contributions to flesh out how we treat these btrfs raid levels.
Thanks for your insightful observations. Even though we have been going for a goodly number of years btrfs has presented many challenges for our aim of a simply NAS with advanced capabilities. The data/metadata split is one we just haven’t gotten around to. This is what you are seeing. Do feel free to experiment with the code and setup your own development enviromnet so you can do so properly but keep in mind that changing raid levels will have far more caveates than it does currently. Hence me sighting the ‘fences’ previously. We aim to be simple and appliance like. As such we will always be less flexibly that command line capabilities. But if we move carefully we can address the majority of what folks actually use/want. And we have had very little interest in separation of data/metadata and zero pull request to date.
Great to have you digging into things here but keep in mind that we are a work in progress, but then all non trivial code is 
.