Over the last few months I’ve been working on this and with eagle eyes one can see small references in comments and the like in a few of my pull requests over that time. Basically openSUSE has a far more advanced and very different btrfs subvol structure to their root. Plus they have their root snapshot / rollback variant of this structure selectable during the install (auto enabled if more than 16GB on root disk I think). Anyway that took an age to work out for Rockstor’s subvol surfacing and the majority of that work was merged in the following:
where I reference the existing CentOS base as legacy.
Also note that within that pr we also account for an erronous backport to Leap 15’s kernel of an unintended upstream btrfs inversion of parent-child columns which also broke a parser of our as the columns were simply the wrong way around, but I managed to account of either arrangement in the end, see:
“Account for upstream parent-child column inversion - group_is_assigned().” in that issue in your are interested. This also brings up the element of potential breakage whenever we change btrfs or kernel version. The upstream inadvertent reversal / kernel backport (at the time) is detailed in our code for reference.
But the above in turn required that our subvol mount system be enhanced as it wasn’t up to the job, hence:
Plus the newer udev (I think) and openSUSE specific treatment there-in broke assumptions we made about the output of udevadm which in turn broke and once fixed rendered unstable our existing by-id name selection hence:
which in turn (and only in part) lead to a regression in out disk activity widget which I’ve just submitted a fix for by way of the pending pr:
And there was openSUSE’s deprication (for security reasons) of ‘–stdin’ on passwd which broke our user password mechanism so this was addressed in:
Then there was the grubby assumption which broke the Web-UI’s ability to retrieve kernel version under an openSUSE source build, which was addressed in:
And then there was the default of no volume label for the system disk in a default openSUSE install which in turn was addressed in the following:
There were and still are a number of build issues which also required attention, one of which was addressed:
which also served to simplify our code.
So all in we have made quite a bit of progress towards rebasing on the openSUSE variants of Leap 15 and Tumbleweed and our subvol Web-UI surfacing of their root subvols was one of the most challenging but I have already achieved a successful replication procedure between a legacy CentOS install and a fresh openSUSE rockstor source build. And given the replication system within tests various other sub systems quite extensively, that was an encouraging win.
But during this time I have endeavoured to serve ongoing requests along with our current backlog of issues but all the endeavour to date has been very productive in routing out otherwise masked issues with out existing code and as such has also helped improve the quality of our current stable subscribers code.
I’m particularly concerned that we don’t break, at least yet, our current ‘in the field’ CentOS users by a major focus for me has been the porting / compatibility work required in our code as it stands. But in this time I have also tried to address glaring current short-comings in Rockstor’s base abilities. All the time trying to ensure that all code works also in the 4 variants of openSUSE, ie each of Leap 15 and Tumbleweed with the 2 rather different takes on root subvol exhibited by the boot to snapshot enabled or disabled variants. Fantastic feature that by the way. I’m sure many Rockstor users will relish the chance to reboot into any one of many prior working (or otherwise) install variants. Makes for a nicer all round experience if one can simply revert to a prior working instance of an entire OS. But so far that facility, although accounted for, currently won’t include the rockstor code itself, again this is from memory and would only involve a reconfig of snapper if required. Although that may also server as a separation between OS and Rockstor code; and in an rpm install setting there is the rpm reversion options for the latter anyway, db migrations allowsing.
Which brings me to my current focus. We have an unrelated build issue re:
which I’m currently working on.
There after, assuming I can get that resolved (I have some pending in test improvements build improvements but have yet to resolve the issue entirely) my next focus in on a btrfs in partition regression that I noted (via in code TODO’s) that I’d like to get to as if we leave too many regression in place we end up going backwards. Plus building on top of the recent fairly hefty improvements in:
Which addressed (bar a single detailed caveat) 3 related issue, 2 of which also depended on 11 other now closed issues. But the remaining caveat still makes it look like this feature doesnt’ work. So I would also like to get around to that remaining caveat expressed in issue:
But also in the mean time we have gained quite a few crucial capabilities, like surfacing missing disk, fs errors etc.
Anyway I think it’s clear that this is a work in progress but we also have a ‘current’ user base to consider, hence the frustrating progress on this.
Interestingly enough I also posted more recently the following by way of a teaser on the openSUSE move:
It’s also interesting that nobody has commented on the lack of their /root subvol being surfaced anymore. With the extensive subvol reworking to accomodate openSUSE’s more grown up use of btrfs it turned out that our prior /root subvol surfacing was an accidental occurrence and was ‘just plain wrong’: so it was removed to return the code to a more proper state where stuff works because it was intended to work .
I suspected this but there is also the SUSE supported transition from Leap 15 to there SLES
with full support. I’m sure that would also be an very attractive option to some of our larger Rockstor users. Plus they employ quite a few btrfs developers and have (obviously) their own kernel developers so can maintain their own btrfs backport to their Leap kernel (Not sure what happens with their Tumbleweed variant). Something that may be overlooked when we move and users still see the 4.12 kernel.
Anyway, as I think is obvious by now, I’m quite enthusiastic about this proposed move and have made I think quite a head way into at least transitioning our code to be compatible with their way more advanced root setup but we have an obligation to also address current concerns, hence my mix of focus of late. But I think we are getting there on:
Ensuring Rockstor’s basic expected / common functionality is in place and working.
Preparing to move to a more appropriate up-steam linux base.
I have been meaning to establish a wiki of the remaining issues re Rockstor and openSUSE but alas as sketched out above, there is much ongoing work that also has to be addressed.
Almost undoubtedly, especially if you are game. But my initial concern is duplication of effort. I have ‘in house’ a sequence of the remaining steps necessary to get build Rockstor master on all mentioned openSUSE variants and it would be a shame for you to have to re-discover them all but I must first focus on getting the build tree back into a ‘works first time’ state. I will then prioritise the wiki so we can steam in with the far more exciting elements of moving to our more obvious home of openSUSE.
I have incidentally also moved my main desktop workstation over to Leap 15 and moved form gnome (Fedora) to KDE in the process so it’s all been quite disruptive but I’m almost settle in again but do have to re-establish at least my prior range of KVM systems upon which I at least developed. These obviously now include a Leap 15 and a Tumbleweed.
I was also quite attracted by their transactional updates install but I think we have quite enough shiny in the near future for the time being.
Please be patient on this and in the mean time if you could establish an existing build setup such as is detailed in:
and familiarise yourself with our existing master branch build, which is also currently our latest stable release version 3.9.2-41 (as of this post) which is now way ahead of our current testing release rpm:
then you will be in a position to assist with identifying and fixing the remaining issue re the openSUSE move as prior to ‘rebasing’ our iso and it’s associated rpm’s we first have to have it build and run as expected on openSUSE as well as CentOS, obviously.
The build OS is essentially a fully updated stable channel Rockstor (to get the production repositories and their move to docker-ce) with at least the associated rockstor rpm removed (although the build procedure is supposed to do this for you but can fail under some manual intervention scenarios).
I’m also very keen on having no regressions at all. In our current user base, stable channel CentOS based, and the proposed new installs (from source or iso install) on openSUSE. This is a tall order but one worth pursuing so if you submit any code changes please bear in mind that I at least would like for them to be non breaking in all of the existing and proposed installs.
One fairly major element of this move to openSUSE, as I see it, is the publicity hit. It should also coincide with the iso being updated, again obviously, so that all non stable subscribers will receive the very latest code which is actually now (in no minor may down to the openSUSE port effort) far more robust that our current testing release. But then there is also the sustainability of Rockstor’s development to take into account, which currently heavily relies on subscriptions. So this is all a very tricky thing that may just end up being another openFiler so any moves we make have to be carefully orcestrated to server the project as a whole which in turn also serves it’s users. Hence the current dropping of testing channel updates, there just wan’t the developer time to go around and we owed those who help support the project a ‘value add’ and that currently is the rpm updates.
Incidentally I did post a teaser image a few months back of Rockstor running on openSUSE but made no mention of what it was. I was hoping to get a bite of interest but alas it has gone unnoticed to date and I now can’t find it!
I do have the following showing a rockstor install from august on a boot to snapshot Tumbleweed install, note the fancy subvol= reference:
This was taken prior to the kernel version / grubby fix indicated above so no new fancy kernel to show off, and I’ve yet to claw may way back to a full KVM set so I’ll post something newer once that’s in place.
Good luck on getting your initial dev setup in place and do take note of my current issue re it’s required ‘hack’ to build. At least until I make some more headway.
I’ll get to that openSUSE port wiki entry shortly but I must first deal with current issue re build and backlog. If you have any build issues please open new thread or pm me as it would be great to get you on board and have this thread as an ‘opener’ on what’s happened so far on that front. I intended the wiki to be a live progress on the remaining hacks required and manual config expected prior to a build with the aim of a clean source build working as expected on only a slightly modified openSUSE install.
Hope that helps you and others on the background progress to date re openSUSE move. As is often the case it turns out to be a lot more complicated/involved than expected. But I do have a plan where all the dots actauly line up .
Oh and as a last note we even have a now rather outdated but hopefully salvageable pull request form @sfranzen on surfacing elements of snapper within Rockstor:
with it’s associated issue:
But that was before I make needed extensive changes to the subvol mechanism so that’s going to be a bit of a major undertaking to get it re-based and working as presumably it once did. But I do hope to get that one re-integrated as there was a tone of hard work that went into it. Thanks to @sfranzen for that and note that it has not been forgotten.
.