Rebasing on to openSUSE?

General
9

@Jorma_Tuomainen and @cferra Hello again to you both and thanks for helping to support Rockstor’s continued development via a stable subscription.

The openSUSE move has been in the viability testing stage for a number of months now and a significant number, but not all, of the stable channel updates have been concerned with adding openSUSE compatibility while maintaining / improving CentOS function. That has been a far more difficult task than just coding out right for openSUSE, as was done prior for our CentOS base. But I am keenly aware that Rockstor’s development is only sustainable (read possible) with the continued support of our stable channel subscribers, along with income from the Donate button within the Web-UI of course. And in this context I’m very keen on making this transition as smooth as possible. But a re-install of the openSUSE based variant is inevitable I’m afraid. However I hope to make further improvements to our Configuration Backup and Restore system before the official ‘newfangled’ openSUSE based installers emerge (new iso) and am planning to write comprehensive documentation, with pictures, on how to transition from the legacy CentOS base to our pending, and still very much in development, openSUSE base. But, as mentioned in my long post above, we also still have basic functionality ‘short falls’ which have to be addressed.

Essentially at the moment and for the last 6 months or so I have had progressively more functional openSUSE Leap 15 (pre release initially) and more recently also Tumbleweed based installs of Rockstor playing a major part in all of my development issues. This has more than trebled my testing burden as openSUSE has boot to snapshot rollback capability which presents root (system disk) subvolumes very differently depending on if it is enabled (default if sys > 16MB) or otherwise) and so there are actually 4 variants I’m trying to cater for just in the openSUSE camp. But I think it’s all going to be worth it as we do away with our bespoke (elrepo kernel-ml add-on) and ancient 3.18 install kernel (rather inappropriate now really) and simply inherit openSUSE’s default kernel maintained in part by some of the btrfs developers. This is a ‘no brainier’ on our move, especially given our past performance on kernel updates: our actions did not match our intentions so the move will ‘fix’ that. Plus I like the many additional options openSUSE presents for image construction / install options. Oh and there’s that boot to root btrfs snapshot thing which I really looking forward to. I.e. one has a working system and an update break it (obviously non of us have ever encountered that :slight_smile:) : our current base offers us little option, and with sever breakage re-installs (of the same OS) have been the quickest / simplest workaround. With a sufficiently large system drive, openSUSE defaults to allowing one to ‘revert’ the root of the os to any number (depending of available space) of prior working examples by selecting them at the grub boot loader. That in itself is worth the move alone. But we also need to address our evident failure to perform timely kernel updates: and this is an obvious area best left to upstream expertise: our linux distro base. This way everybody wins, we get to spend more time on the actual Rockstor code and the progressively more numerous Rockstor users will get to see the rather under represented glory of the openSUSE / SUSE distros and release tools. Plus a rather under reported element of Leap 15 is that it now has binary compatibility with SUSE’s SLES. That will be very important in at least a few mission critical circumstances.

OK initial (at least dev focused) spiel over:

They are and have been continuing during this development, and in no small part because of this development; CentOS cannot be our future: they have made this clear on their btrfs stance. OpenSUSE defaults, as we have done for quite some time, to a btrfs root and as a result can offer features no other linux distribution can.

For an as yet undetermined time I’m at least hoping to maintain code compatibility with CentOS but the creation of these rpm’s may well have to transition to our new build system and there may well be blockers via diminishing returns on maintaining our CentOS rpm releases. Especially given we occasionally receive low level fly by contributions that may well not have been tested on what will then be 5 different install bases and that’s a lot to ask for a casual contributor. Plus, if we do this right, all will automatically benefit from moving to the openSUSE base.

Short of it is this is a messy arrangement, but we should come out better overall.

Thanks for your most welcome encouragement, this mirrors my feelings on the matter. It’s also a major bonus that SUSE helps to fund openSUSE but remains independent from them; they also employ a number of btrfs developers.

The stable channel subscription funding model is a value add to our ‘backers’: more frequent Rockstor code updates. And a subscription, along with it’s associated activation code, is tided to the motherboard used during the install, ie the output (run as root) of ‘cat /sys/class/dmi/id/product_uuid’ and so any re-install using the same motherboard (bar the caveat of a few known non unique product_uuid’s) should result in the same activation code working as before. As it’s the exact same Rockstor code (bar some command paths) running on the re-install it was on the prior install: even given the move to an openSUSE install base. It is in everybody’s interests that this be maintained. But as I say we arn’t yet at that (rpm) stage and I’m currently still working on achieving the cross distro (CentOS, openSUSE Leap15/Tumblweed) functional equivalence.

  • Config save and retrieve to a client machine: Configuration Backup and Restore.
  • Power off & physically detach (unplug) all data drives (optional but highly recommended).
  • Re-install on same motherboard, and ideally to a different system disk but not necessarily.
  • Enter prior installs activation code (it’s the same motherboard so will have the same appliance id).
  • Apply update and reboot when done; if all is ok power off.
  • Re attach all data pool devices.
  • Import pools (btrfs volumes), shares, clones, snapshots (btrfs subvolumes).
  • Import prior saved config.
  • Carry on as usual, the Rockstor code is and will be essentially identical.

Given the above upgrade path:
settings yes with proviso that it’s currently limited, Pools, shares, snapshots yes, rock-ons possibly not as we currently don’t have a CentOS re-install transition path for them either.

Essentially I want to preserve our current re-install capability across to the as yet non existent openSUSE install rebase. Note that all contributions on improving the config back and restore are of course welcome as this would inevitably ease this transition. I will try to take another look myself also but this is obviously a busy time. And we have mounting pressures to address our current short falls re kernel btrfs updates which in this context may not be time best spent. Rockstor is a small but dedicated team, but we also have quite a high contributor count for our age and niche, especially given the projects code size / scope. We have also had quite a number of first time GitHub contributors so I think bodes well for us.

New iso. Which of course also means that we finally get our now far better stable code into the hands of the more casual users. This will ultimately be a good thing and presumably, if it all lines up, we carry on as before: ie value add in Rockstor code update frequency for the subscribers. But with a far better introduction to the project for our then new installers as they will then get to run the now far more developed recent code.

Inevitably much improved, especially for the installer. Our current outdated CentOS installer has kernel 3.18 and is a ‘job’ to update. OpenSUSE’s image build tools are fairly fantastic and far ranging so new iso’s should in the future be more common and varied in option (possibly). I’m also hoping to move to a system disk image transfer type install option as a default (much work to be done on this as of yet) as there is then almost zero room for an ‘incorrect’ or incompatible install and should open up Rockstor to a lot more people and make re-installing a far faster and easier affair.

Hardware alterations (different system disk if used) carry with them inevitable risk. But as the Rockstor code itself will be unchanged (as far as we can make it work that way) the risk should be minimal, especially if the above advise is observed (ie disconnecting data drives entirely). And if one uses a different system disk for the openSUSE install then if it doesn’t work our right away (teething problems) then one can simply revert to booting from the prior CentOS based Rockstor install which shouldn’t pick up from where it left off. Hence the recommendation to use a new system disk for this new OS based Rockstor install, which to be clear doesn’t as yet exist; but I’d like for it to and addressing these questions is a priority as it is likely only to come to fruition via the continued support of the stable channel subscribers such as yourselves.

Only in the same case as we currently have where the motherboard in question has a non unique product_uuid.

There are still many details to work out but where there is a will I am hoping there is a way.

Any ‘would be’ code contributors / debuggers might like to take a look at our current (master branch) configuration save and restore mechanisms as they will, during this transition, receive quite a work out, so it would obviously be best if we do what we can to address their bugs / short falls prior to this move. None of that code is distribution specific, working mainly at the database save and restore level, so if any changes work on our current CentOS base they should also work equally well on our proposed future OS base of openSUSE.

The rockstor-core code is licenced under versions of GPLv2 or later. Our various and numerous rockstor-jslibs and the like have their own particular licences but are all never the less open source. The subscription system is not a licence grant of any sort, just an attempt to achieve a sustainable development model for Rockstor by way of a value add exchange. Although the Update Channels Testing Channel section is, at least for the time being, currently out of date, the intro/history and Stable Channel sections remain accurate bar the reversal of update frequency of course. We also have no contributor licence agreement which is important to some, including me.

Thanks for your pertinent questions and I hope this helps to ally any worries on the stable channel subscription plans. At least initially this looks to be the way to proceed but ideas, as well as donations, are welcome as many open source projects such as ourselves struggle to achieve sustainability while simultaneously making on-going efforts to supply a need.

1 Like
10

Is CentOS still going to be supported or receiving more up-to-date kernel, or any other (security) updates from 7.6 while OpenSUSE rebase is being worked on? Leaning towards renewing my subscription but reinstall is always a hassle.

Not fond of suse and my day job is keeping tens (between 50 and 100) of different (95% CentOS 7) servers up-and-running, that’s just what I’m used to :slight_smile:

11

@Jorma_Tuomainen

From the Rockstor code perspective my intention at least is to maintain code compatibility with CentOS, but ultimately this whole venture is to improve the Rockstor experience by users not relying on our already failed attempt as regular kernel updates for example. So yes the rpm updates are inevitably going to stop on the CentOS platform.

That would be nice but we have failed on this one for a while now so this may also not happen (see moving our distro base so they do it for us in expert and timely fashion)

Yes, bar the kernel (copied from elrepo), btrfs-progs, and netatalk (AFP deamon), all other packages receive updates as per usual from the upstream CentOS repositories, ie from a package point of view look at the following in a stable channel Rockstor install:

yum list installed | grep '@rockstor\|@Rockstor-Stable'
btrfs-progs.x86_64                 4.12-0.rockstor                @rockstor     
docker-ce.x86_64                   17.09.0.ce-1.el7.centos        @Rockstor-Stable
kernel-ml.x86_64                   4.12.4-1.el7.elrepo            @rockstor     
kernel-ml-headers.x86_64           4.12.4-1.el7.elrepo            @rockstor     
netatalk.x86_64                    5:3.1.11-0.1.1.el7             @rockstor     
rockstor.x86_64                    3.9.2-42                       @Rockstor-Stable
rockstor-release.x86_64            3-9.23.el7                     @rockstor

And the netatalk was only there to address a short fall on CentOS’s part.

So it’s really only those packages, the rest of the system is as per a regular CentOS install and so my understanding is that it will simply update in place as it always has done, pertaining to their support time lines anyway.

I’m personally not fond of CentOS (slow to adopt technologies), but then at another point I wasn’t fond of SUSE (no openSUSE at the time) as yast would mess with ones config files. However I’d like to say that bar a few inevitable hick-ups CentOS has been good to us. But simply put their direction is just not ours and we, as a small team with a large project, obviously need to rely on our upstream, especially and evidently for our kernel and filesystem updates. Plus of the 3 Enterprise associated variants openSUSE/SUSE is an obvious choice for us. I particularly like openSUSE’s community lead governance (only one board member is, or can be, a SUSE employee) and the fact that openSUSE survived a ‘quiet’ time for SUSE back there. Can the same be said of the now less independent CentOS?

Too right. But I don’t see a way around it. But I’m keen on moving to a far more ‘lean’ image based install so hopefully that will be a form of compensation (more on this once we have our code in order) and move to exploring / testing the install options. And hopefully we can also offer a variety of install options, ie ARM based for example.

Yes, distribution preference often comes down to that, and in fact one of my first work distributions was the original Red Hat (I chose it) before it became RHEL some way down the line, and dating from a job I used it on at the time it was November 2000 “17th International Diabetes Federation Congress” held in Mexico City. I did the registration sever database, which at the time had never run on a linux machine before, according to the developers (linux was still quite new back then). Access to the db backend from the windows clients was over SAMBA (yes don’t ask) and I had to be really really careful with file permissions and timing as the proprietary db would lock if any were out: obviously suspecting/claiming foul play of course!. I ran it across a couple of independent servers so we had complete system failure over capability. This would make it around version 6.2/7.0 by the looks of it (Oh the glories hailed for the shiny new 2.2 kernel of the time :slight_smile: ). But I’d played with it from around 5.1/5.2 (1998) if I remember correctly. At around that era I moved my mother and my mother in law and father in law’s desktop machines over to whatever was the current version and they all saw a good few year of service. I’m a bit of a linux fan as it goes.

Hope that helps.

1 Like
12

Based it on CentOS will make RockStor something on the net, a dedicated server. It is not mainstream software. If you want this to be distributed, you should change to Ubuntu / Mint - that is where all the kernel and device management code is done. A very good reason is “apt” - allowing release and version management at all installations. I cannot use Rockstor as it is, then the CentOS base will be considered and new platforms like DeepIn is not here. But router and network switches are based on CentOS. The AV security is based on Ubuntu / Mint - all the “Bleedin” was on Mint. But: The difference is in how it looks, not how the disk works. Ubuntu / Mint walked down a sily route with the “ssl”, but they have miles better exposure to the DMA arbitration, device drivers. They dropped KDE because nobody used it! DeepIn is pretty close.

13

What, exactly, are you trying to say? All I can take from it is that you seem to think that switching to an Ubuntu/Mint base is the only way forward, or something.

14

It is not The Only Way forward, but to e more than just a few, you should consider making a branch that is based on the distribution with the most users. Here you get the latest hardware updates and the kernel is tested on these. I have said the same to DeepIn, it is better to stay together like Ubuntu and Mint has done.
You have a valuable collection of tools and management software that I would love to present to others. The routers and TV box hardware runs Linux, and can run disks / mass storage. This is used to store tv shows for later viewing, but why not Rockstor and offer home cloud service? All photos is then available at home, all the private videos. This is where most of us use the media, they are just files - like my Word document and spreadsheets. Your decision to control the platform makes this impossible. But package the scripts and binaries as “apps” and RockStor will get a much bigger audience.

15

@knut,

I’m as big a fan of Debian based distros as anyone (though I’d choose Debian itself, not it’s derivatives) however to address some of your points:

This is a relatively unknown statistic. I’m assuming that you’re basing this on data from DistroWatch, in which case it should be noted that this is calculated based upon page hits from desktop systems. Mint and Ubuntu are primarily desktop systems.
This is not what Rockstor is designed to be. The idea is to turn a computer into a NAS appliance, not a desktop computer.

And Rockstor can do those things as well, by itself.

This is exactly what I have setup. Rockstor running Plex for media, Owncloud for document sharing, etc.

Rockstor is the platform. That it’s built on CentOS or OpenSUSE is beside the point.
The idea here (re-iterated) is to run Rockstor like an appliance.
I’ll concede your point the day you install a toaster into your router.

Rockstor is a BTRFS NAS. Arbitrarily installing it onto a system it doesn’t understand will not work.
In the same way you cannot arbitrarily install any linux flavour of your choice and then install FreeNAS, OpenMediaVault, Synology DSM, etc.

The fact that you install it yourself does not change the fact that Rockstor is designed to be a device, not a program on an arbitrary device.

16

Your attitude is wrong. RockStor will be another comma in a sentence in history.
I have made file systems - you have not.

17

In light of the fact that this is becoming a somewhat heated argument, it should be pointed out that I am not from the Rockstor team, and my opinions/arguments do not necessarily represent that of the development team or the greater Rockstor community.

In your opinion. I don’t see why this needs to be a personal attack.

That would actually be a pretty good rating, many things go by that don’t equate to near that.

Really? Which one(s)? - I’m generally curious.
You’re right, I haven’t built any filesystems (though I wrote a FAT16 ARM driver once).
Regardless, your argument is not even about filesystems (near as I can tell anyway).
It seems that you’re arguing twofold:

  • For Rockstor to be a portable application rather than an appliance level OS.
  • For Rockstor to be rebased on Ubuntu/Mint.

For the first bullet point, this is simply not the purpose of the project. I agree that it would be a lovely idea to have a portable UI such as Rockstor, but without control over it’s entire environment, it needs to be built in a way to understand and support a multitude of different environments.

This is compounded further by the fact that Rockstor’s native filesystem (BTRFS) is managed in completely different ways to traditional linux filesystems.

The development resources to support this would be immense, and this is developed by a very small team.

Regarding the second point, I again point out that this is based upon an invalid premise that Ubuntu/Mint is the de-facto default Linux OS.
Just because it’s favoured among desktop users, this does in no way make it a de-facto default.
Also, Ubuntu and Mint are both forked from Debian and rely on Debian’s upstream.
This leads to the same challenges that Rockstor already faces, namely provisioning up-to-date Kernels due to the long release cycles.

Noted of course that at least with a Debian base you can select to be on the testing or unstable branches for quicker updates, however in my experience (Primarily a Debian since Hamm in 1998) the testing and unstable branches are dangerous to run as an appliance, as they’re quite prone to dependancy issues on the default repositories.

OpenSUSE tends to have shorter release cycles, meaning things like the Kernel is updated quite frequently. This is particularly useful to Rockstor, as it’s built on BTRFS which is actively developed and frequently updated.
Using an OS with a more frequent release cycle allows the developers to use the bleeding edge BTRFS features supplied by newer kernels.

All that said, I would suggest that if you don’t like where the project is headed, you are welcome to fork it and go your own direction, or use a different project.

1 Like
18

In my opinion there is no real need to keep the CentOS Branch alive as long as no kernel updates would come anyway. So probably it would be better to just fokus on getting that opensuse version out, even if it would come with some hassle for migrating (perhaps complete new install). But delaying the migration because of getting a live migration working is perhaps not worth the time as you just fall back even more kernel versions with a lot of btrfs fixes.

5 Likes
19

And here we go:

It’s a bit rough but at least it’s ready (exists) now.

Hope that helps those interested in helping with this development.

Please remember that any proposed changes should not break existing CentOS installs . At least for the transition period.

2 Likes
20

Hello,

I got pointed out to this post, but I could not find why not to switch to Fedora? I mean, btrfs support is there, snapper is there, there is even integration between grub2 and snapper.

Is there anything Fedora community could do so that you will consider building platform on top of it?

21

Not a Rockstor developer, but:

  • Rolling release
  • YaST/YaST2/WebYaST
  • Suse Studio
  • BTRFS by default
22

Thumbs up from me on the re-base, fwiw. Was already planning to migrate my home server/NAS from Debian Stretch to Tumbleweed, but I think it would be fairly practical to see what happens here.

23

@vfrex Welcome to the Rockstor community and thanks for the encouragement. I’ll pop in an update while I’m here.

Yes we are making progress, bit by bit, and my latest source build a few days ago found a couple of breaking upstream changes on postgres and docker but we have a work around for both, the upstream postgresql issue work around was added to our already referenced openSUSE dev notes and status, thanks to @jcdick1’s recent experiments along the same lines, and the docker change/workaround is documented in the following issue opened at that time:

https://github.com/rockstor/rockstor-core/issues/2032

Just need to rename a file for now and docker is back up and running again.

All rather alpha at the moment but if you are interested in building from source it is possible:

latest-so-far-with-plex-installed

and we have provision in place to transition sufficiently current source built installs over to rpm installs once we have the back-end infrastructure in place to distribute them: closely related to my ongoing task re Rockstor as it goes. Just note that source installs have no provision for updating as they necessarily wipe the settings db (postgres) on each build: given in development there is no set point from which to establish db migration points.

Hope that helps and if you do fancy having a go consider contributing to @jcdick1 recent thread on installing via source build on Tumbeweed as that is the current forum focus point for difficulties in that area. I’m trying to keep this thread for development progress reports.

1 Like
25

I don’t think so myself as we are not yet at feature parity in the “Built on openSUSE” rpms, and we also still have quite a few bugs / failures in those rpms that don’t exist in the CentOS offering.

Not really, especially if you are already running a Stable update version of the CentOS offering as then one should be able to backup ones config on the CentOS variant, install using our as yet unreleased openSUSE Leap15.1 based installer (less than 5 mins on years old hardware), import the same pool and apply the config backup. See Configuration Backup and Restore in the docs. This very process can be seen working as of now for both my and @Flox’s tests of their recent improvement to this config save/restore mechanism:

https://github.com/rockstor/rockstor-core/pull/2100

that was released in version 3.9.2-52:

@Flox has also done another improvement to this system since so we are trying to smooth the way for this transition. Plus if one has not created any additional shares on the system drive then the system / data separation is maintained, which in turn simplifies any re-install process. I’d like to remove the ability to have user shares on the system drive all together but in some settings this ability is ‘just the job’, so maybe we need to add some additional warnings to our existing Web-UI warning re limitations of system drive shares.

Hope that helps.

1 Like
26

@phillxnet Might be you or point me to someone else if not.
I have the second HP server finally up and working. So if anyone would like I would be more than happy to slap on the new “Alpha/Beta” of Rockstor and help in any little bit I can.

I can not code so I can just do some testing in other ways if you want.

D…

1 Like
27

@CJ_and_Darren Hello again.

Thanks for the offer. It’s actually getting close to the time when it would be appropriate for more general testing, but currently it’s still pretty much in the developer only stage as what we really need is direct assistance with the fixes themselves. It’s still a target rich environment :slight_smile: . Once we have the most obvious stuff done then I or @Flox will annouce this and that way we can get on with the final refinement via wider testing with the aim of achieving feature parity with our CentOS variant and then finally releasing the ‘Built on openSUSE’ Stable channel rpms. Can’t wait myself personally, but all in good time.

I’d say wait until at least 3.9.2-54 (next release) as there after we expect samba and email notification to actually work. From there on it should be down hill. Thanks to forum members @Flox, @def_monk, @gaspode, @Marenz, @luke, @Psykar, and @tyukh for their efforts with ongoing recent bug squashes on the openSUSE side.

But by all means do take another look at our companion post re the now released ‘Built on openSUSE’ testing channel. I’ll update this thread soon on the missing 10 months of progress since my last post here. But our 3.9.2-* Stable channel announcements have served this purpose by flagging all openSUSE releated issues fixed in the interim.

Thanks again for the offer and maybe there is some utility in just getting Leap15.1 on that rather fancy HP server you have ready for the pending rpm updates. Plus we are now down to just a few tweaks from default Leap15.1 install, prior to testing repo add, so I say give it a go but there are just not enough core developers to assist with this process without in turn holding up the larger mission. Hence the request for developers. But if you are game I think 3.9.2-54 may be worth a look. Oh and the current 3.9.2-53 should be able to update in place, via the Rockstor Web-UI to -54 when it comes out. Just in case you are really eager. But remember this is not a supported release, we need the support of others actually, hence the linked post stressing developers. But I think you get the idea.

Great to have the early adopters on board with this, it can only help. But again folks, please appreciate we are a pretty small team, so ideally, as the linked post states, we are mainly in need of the actual fixes, rather than the reports of sub system failures as they are, at this stage at least, fairly easy to identify.

1 Like
28

Just a quick update by way of copying in here what has been noted as fixed from our 3.9.2-# Stable channel updates announcement thread that pertains to openSUSE specifically:

3.9.2-49

2.9.2-50

3.9.2-51

3.9.2-52

3.9.2-53

This list does not include numerous config save / restore improvement that will also help in the transition, these have been almost exclusively contributed by our intrepid forum moderator and core contributing developer @Flox . It also doesn’t include our regular improvements which in almost all cases improve stuff across our currently 3 distro targets.

I may have missed some here but that’s roughly the gist of the “Built on openSUSE” only orientated fixes / improvements.

Getting there folks.

1 Like
31

Another quick update taken from our 3.9.2-# Stable Channel announcements thread pertaining to our ‘Built on openSUSE’ testing variant:

3.9.2-54

  • -Source package default smb.service in “Built on openSUSE” variants. Fixes #2127 @FroggyFlox
  • -[openSUSE] fix postfix config re ipv4, tlsmgr, & CA file settings. Fixes #2132 @phillxnet

and:

3.9.2-55

[openSUSE] adapt to shellinabox package differences. Fixes #2006 @phillxnet

Please see the master thread for more details:

Just copying in here the ‘Built on openSUSE’ related changes thus far.

Bit by Bit.