I am running a Server for different Home-Automation&Theatre tasks.
This server runs fine (under ubuntu) and I don’t really want to change it. Nevertheless, I would like to use Rockstor. So, I would have to install another Server with the associated cost.
Alternatively, I could use a virtual machine for Rockstor, but this has associated overhead.
Wouldn’t it be a good idea to put Rockstor in a Docker Container? This would make testing Rockstor easier and it would allow using Rockstor as an ‘app’ on an existing Server.
@henfri Thanks for sharing your ideas. Rockstor as a docker container(or as a set of containers) is an interesting idea and it’s something I want to do for our development environment to start with. It could make our development cycle faster. However, since the container would have to have access to physical drives(or virtual ones), host networking etc… operationalizing this and supporting it for end users is a bit too much to commit to at this time. This get’s even more burdensome with Rock-on’s which are docker containers themselves. So, in summary, Rockstor doesn’t seem to be ready for being a docker container itself at this point. But let’s keep this discussion alive so we can revisit it when we have the bandwidth.
thanks for your feedback. I understand what you are saying, but if I understand correctly, both access to physical drives and docker in docker are no problem:
With this option (-privileged), docker can access /dev/.
the only real problem i can see is the kernel, as rockstor uses btrfs the kernel has to be the most recent version for the greatest bug fixes and improvements. As long as you run it on something like coreos with some mostly up to date kernel you should be fine, but a simple ubuntu server/or whatever might need manual kernel builds by the user (underlying layer, the rockstor image cant access it). So support is getting more complex as every install in docker might use different kernel versions interfering with different things in the image. You could create kvm/qemu vms for this purpose, they use a bit more ram as they dont share the kernel and some other things docker shares, but you get your unified layer in kernel perspective. Enabling vt-d/iommu for your disk acces in rockstor gets you greater performance as rockstor gets the controller passed-through, but you might need a dedicated one here. Enabling nested virtualization support for this vm should get your rockstor docker container up and running with hardware assisted virtualization support.
I’ve taken this a bit further and got something running well!
Then of course notice that migration to opensuse is on the way. I’ve started that in a separate branch, working from Built on openSUSE dev notes and status
Did notice that the dev notes topic above doesn’t mention that you need to install firewalld as well. I may run into more… we’ll see. (@phillxnet - can you edit to add firewalld to the list of “Packages for Rockstor function?”)
This is a super exiting project, and thanks for building on @bigfoot90 start in this area and sharing your progress.
If this gets more traction we could consider adopting it as an official Rockstor repository. That might help to focus efforts on this particular implementation.
Of course there is the obvious fractal self hosting question of “Fancy submitting a Rock-on”, ie see our rockon-registry Readme section: How can I add my own app?
There are a few caveats but not that since this project was started by @bigfoot90 we have since gotten the ability to pass arbitrary devices through to a rock-on, ie potentially block devices maybe. See the @Flox contribution in the above Readme of Devices object and their now merged pull request of:
against issue:
Not the use it was intended for but still. @Flox, I’d be interested in your input on this ‘alternative’ use of your recently added device capability.
This might for part of how this Rockstor inception arrangement might take useful form. There are of course a few (read many) potential nightmares (multiple Rockstor’s managing a single pool = very bad) but then Rockstor is not best suited to dockerisation anyway, given it’s direct, and expected sole, block device control expectation. But it’s an interesting project.
Thanks for contributing your progress. Nice.
If you do embark on the inception Rock-on could you please give it a very large warning re catastrophic corruption potential if not allocated an otherwise un-managed block device. To be frank this is a fairly hairy project, but I don’t see, given a large warning in the description, why it shouldn’t be entertained. At least in the long run anyway. Maybe we could even scan currently installed Rock-ons for such a device mapping and block the host Rockstor from using / assessing that device. We have a role system that could effectively do that. And like wise only allow devices passed to this inception Rock-on that are not currently managed by the host Rockstor. Interesting for sure.
Lets see how this effort is received by the Rockstor community at large.
Thanks and sorted. Leap 15.1 (now in beta) dropped the firewalld package on server type installs, so noted as such in the referenced doc.
And Thanks again for considering our future base of openSUSE. At least then we will have a more viable btrfs base and potential support escalation to SUSE themselves. Both major influences in our move given that as from openSUSE Leap 15.0 there is now binary compatibility to SLES (Suse Linux Enterprise Server).
Worth noting - I wasn’t envisioning this as being a rockon, but rather a way to run rockstor on a non rockstor host (my testing was an ArchLinux host).
My biggest concern at the moment is things inside the container that could affect the host. Block device access obviously is one - but in general I wouldn’t expect Pool devices to be in use elsewhere.
However, I got a bit of a surprise when I realised that because docker uses btrfs and therefore the root filesystem of the container is a subvolume of the host’s filesystem, that when the rockstor setup enables quotas, it’s enabling quotas for the hosts filesystem as well… due to the large number of snapshots my host has it wasn’t the greatest
I might see if it’s possible to come up with a list of capabilities combined with passing specific devices/drives down that would allow this to run without privileged, which would have prevented this issue *ponder
I believe running an nfs server inside docker is a problem and you need to pass certain privilege flags to docker to allow it to happen. Another issue is that docker, docker-compose and docker swarm all seem to do this slightly differently (and im not sure you can in docker-swarm).
There is even a user-space nfsd, but even that needs privilege.
if not allocated an otherwise un-managed block device. To be frank this is a fairly hairy project, but I don’t see, given a large warning in the description, why it shouldn’t be entertained. At least in the long run anyway. Maybe we could even scan currently installed Rock-ons for such a device mapping and block the host Rockstor from using / assessing that device. We have a role system that could effectively do that. And like wise only allow devices passed to this inception Rock-on that are not currently managed by the host Rockstor. Interesting for sure.
