Rock-on framework implementation

Note: this document will be split in several posts in order to fit within characters’ limit. Make sure to browse the second post as well for more information :wink: .

Preambule

This is a wikified post documenting how Rockstor implements the rock-on framework, and should thus be considered a live document expected to be updated as necessary. As a user documentation already exists on how to use the framework (see link below), however, this write-up intends to first summarize the overall rock-on framework concept, describe its implementation from a developer’s perspective, and centralize the discussion on its current state and future evolution. By combining these different elements into a single document, this post will thus attempt to provide an integrative view of the current state of the rock-on framework and bring together everybody’s expertise and ideas to foster a continuous, coordinated, and coherent improvement. In this context, recommendations and suggestions are more than welcome!

While many details will be provided, it is important to note that a focus will be placed on simplicity in order to provide a general description of how the different processes work and interconnect. As a result, we will not go into a line-by-line description of all steps, and only the code that directly mediates each step of interest will be shown—the rest will be masked ((...)). For details, however, we will provide the reader with a link to the full code.

For a general description of the rock-on framework, please see the related section in Rockstor’s documentation:
http://rockstor.com/docs/docker-based-rock-ons/overview.html

Implementation overview

In its essence, a rock-on corresponding to an ensemble of settings in a JSON format. Hosted on a public Github repository (rockon-registry), they contain all necessary information regarding the underlying docker containers and their configuration, as well as the rock-on’s metadata. While all parameters are stored in Rockstor’s database following a “key:value” structure, some parameters, such as volumes and environment variables, for instance, will have their value defined by the user during the rock-on install process. Indeed, these settings are surfaced in the webUI during the install process to be customized by the user. Once all these settings are filled by the user, Rockstor updates their values in the database before starting building up the docker commands to run the underlying container(s).

Once the user clicks the “Submit” button at the end of the rock-on install wizard, Rockstor fetches all parameters linked to the given rock-on and builds a corresponding docker run command for each container defined in the rock-on. Once the docker run command is built, Rockstor triggers it as a background ztask surfaced in the webUI as “Installing”. A successful rock-on install thus results in the creation and start of its underlying docker container(s) using the settings defined in the JSON file and by the user during the install wizard.

Finally, toggling ON and OFF a rock-on from the webUI simply translates into starting and stopping the underlying container(s), whereas uninstalling it will remove it/them.

Implementation details

In this section, we will describe in more detail the different aspects of the rock-ons framework and provide heavy references to the corresponding parts of Rockstor’s code. While this is not an exhaustive documentation of the underlying code, this will hopefully help serve as an illustration of Rockstor’s logic and offer starting points and guidance into the specific parts of this framework.

Main items of interest

Database

All rock-ons-related information is stored in the storageadmin database under dedicated but interconnected models (depicted in the picture below).

As you can see above, rock-ons-related models are centered around the DContainer model rather than the RockOn model. This is simply due to the fact that the container is the base unit of the docker environment and thus represents the best entity to link all information. As a rock-on can include multiple containers, however, the RockOn model is the unit used by Rockstor to surface this information to the user to interact with.

Accordingly, the RockOn model mostly includes elements surfaced to the user in the webUI:
link to code

class RockOn(models.Model):
    name = models.CharField(max_length=1024)
    description = models.CharField(max_length=2048)
    version = models.CharField(max_length=2048)
    state = models.CharField(max_length=2048)
    status = models.CharField(max_length=2048)
    link = models.CharField(max_length=1024, null=True)
    website = models.CharField(max_length=2048, null=True)
    https = models.BooleanField(default=False)
    icon = models.URLField(max_length=1024, null=True)
    ui = models.BooleanField(default=False)
    volume_add_support = models.BooleanField(default=False)
    more_info = models.CharField(max_length=4096, null=True)

The DContainer model, on the other hand, is very simple and the only critical information it stores is its name:
link to code

class DContainer(models.Model):
    rockon = models.ForeignKey(RockOn)
    dimage = models.ForeignKey(DImage)
    name = models.CharField(max_length=1024, unique=True)
    launch_order = models.IntegerField(default=1)
    # if uid is None, container's owner is not set. defaults to root.  if it's
    # -1, then owner is set to the owner of first volume, if any.  if it's an
    # integer other than -1, like 0, then owner is set to that uid.
    uid = models.IntegerField(null=True)

As noted above, this is due to the fact that the container is a base unit of the docker environment and its model thus mostly represents a hub of all specific information applied onto it. This specific information is thus stored in dedicated models keyed by container and includes:

class DImage(models.Model):
    name = models.CharField(max_length=1024)
    tag = models.CharField(max_length=1024)
    repo = models.CharField(max_length=1024)
  • DPort: list of all ports defined in the JSOn file, and their default host mapping.
    link to code
class DPort(models.Model):
    description = models.CharField(max_length=1024, null=True)
    hostp = models.IntegerField(unique=True)
    hostp_default = models.IntegerField(null=True)
    containerp = models.IntegerField()
    container = models.ForeignKey(DContainer)
    protocol = models.CharField(max_length=32, null=True)
    uiport = models.BooleanField(default=False)
    label = models.CharField(max_length=1024, null=True)
  • DVolume: List of all volumes and their share mapping. Note the uservol field enabled during post-install customization (see below), as well as its connection with the Share model, populated when the user selects a pre-defined Rockstor share from the drop-down menu at rock-on install.
    link to code
class DVolume(models.Model):
    container = models.ForeignKey(DContainer)
    share = models.ForeignKey(Share, null=True)
    dest_dir = models.CharField(max_length=1024)
    uservol = models.BooleanField(default=False)
    description = models.CharField(max_length=1024, null=True)
    min_size = models.IntegerField(null=True)
    label = models.CharField(max_length=1024, null=True)
  • DContainerLabel: Used during post-install customization only, this model stores simple key:value mappings for docker container labels.
    link to code
class DContainerLabel(models.Model):
    container = models.ForeignKey(DContainer)
    key = models.CharField(max_length=1024, null=True)
    val = models.CharField(max_length=1024, null=True)
  • DContainerEnv: This model stores simple key:value mapping for a container’s environment variable(s).
    link to code
class DContainerEnv(models.Model):
    container = models.ForeignKey(DContainer)
    key = models.CharField(max_length=1024)
    val = models.CharField(max_length=1024, null=True)
    description = models.CharField(max_length=2048, null=True)
    label = models.CharField(max_length=64, null=True)
  • DContainerDevice: This model stores simple key:value mapping for a container’s device binding.
    link to code
class DContainerDevice(models.Model):
    container = models.ForeignKey(DContainer)
    dev = models.CharField(max_length=1024, null=True)
    val = models.CharField(max_length=1024, null=True)
    description = models.CharField(max_length=2048, null=True)
    label = models.CharField(max_length=64, null=True)
  • DContainerArgs: This model stores command argument(s) to be added to the docker run command, as defined in the JSON file.
    link to code
class DContainerArgs(models.Model):
    container = models.ForeignKey(DContainer)
    name = models.CharField(max_length=1024)
    val = models.CharField(max_length=1024, blank=True)
  • DContainerLink: This model store simple source:destination mapping used for docker container links.
    link to code
class DContainerLink(models.Model):
    source = models.OneToOneField(DContainer)
    destination = models.ForeignKey(DContainer,
                                    related_name='destination_container')
    name = models.CharField(max_length=64, null=True)
  • ContainerOption: This model stores the values of specific options defined in the rock-on’s JSON file. This information is not surfaced to the user.
    link to code
class ContainerOption(models.Model):
    container = models.ForeignKey(DContainer)
    name = models.CharField(max_length=1024)
    val = models.CharField(max_length=1024, blank=True)

As you may have noticed, one last model, DCustomConfig, differs from the others as it exists at the rock-on level rather than at the container level. This model has a very specific use for the installations of two rock-ons (owncloud, and openvpn) for which dedicated scripts exist:

Files

Three main files interact with these different database models:

  • rockon.py: fetches the list of all available rock-ons in the metastores (remote and local) and fills information from JSON definition files into the database.
  • rockon_id.py: handles specific operations on a given rock-on: install, uninstall, update, start, and stop.
  • rockons.js: responsible for all UI operations, such as list listing all available and installed rock-ons, and installation wizard, and post-install customization.

In addition, related helpers are located in rockon_helpers.py.

Rock-ons catalog (list)

To get the list of available rock-ons, Rockstor uses two different sources (termed “metastores”): one remote, and one local. Both are defined in Rockstor’s django settings:
link to code

ROCKONS = {
	'remote_metastore': 'http://rockstor.com/rockons',
	'remote_root': 'root.json',
	'local_metastore': '${buildout:depdir}/rockons-metastore',
}

Remote and local metastore

The remote metastore includes a list of all rock-ons in JSON format (root.json) as well as the JSON definition file for each rock-on in the root.json list. All files can be found in the Github rock-on registry.

Rockstor first fetches information from the remote metastore, loading the JSON definition files for each value in the root.json file. It then proceeds in doing the same for the local metastore.
link to code

This procedure is triggered in rockon.py through the call to _get_available():

    def post(self, request, command=None):
                (...)
                rockons = self._get_available()
                (...)
                for r in rockons:
                    try:
                        self._create_update_meta(r, rockons[r])
                    (...)

As can be seen above, Rockstor then proceed with updating the database information for each rock-on through _create_update_meta().

Rock-on definition fetching

The entire process is taken care of by _create_update_meta().
link to code

First, a set of default values is used to create a RockOn model instance for the given rock-on. This instance is then completed and updated with the contents of the rock-on JSON definition file. This includes rock-on-related information such as its description, link to project website, version, presence of webUI, more_info section, etc…

Then, the “containers” key in the rock-on JSON definition file is parsed to list the names of all the containers included in the rock-on. Each value (container) is then parsed to extract the container’s settings and update the corresponding models in the database accordingly. While the process is fairly straightforward, there are several important points:

  • If no information regarding the docker image’s tag is found in the JSON file, the default is set to latest.
    link to code
            defaults = {'tag': c_d.get('tag', 'latest'),
                        'repo': 'na', }
            io, created = DImage.objects.get_or_create(name=c_d['image'],
                                                       defaults=defaults)
            co.dimage = io
  • When filling the default host port for a given host:container port mapping, Rockstor checks whether the host port is already set to be used by other available rock-on (installed or not). If it is indeed the case, it will simply set the default to use the next available port. This explains why the default port presented to the user during a rock-on installation wizard can differ from the one defined in its JSON file. Notably, this check is only performed within ports defined in rock-ons’ definitions and is thus not aware whether a port is actually in use by another service on the local system. This is especially important to keep in mind for users who are running other applications on their machine as conflicts can then arise.
    link to code
                    def_hostp = self._next_available_default_hostp(p_d['host_default'])  # noqa E501

Display in UI

Main file of interest: rockons.js

With all rock-on information in the database, Rockstor’s front-end can simply fetch this information and present it to the user. This is done via the Backbone collection RockOnCollection, which in turn refers to the /api/rockons call to get and return the list of all rock-ons from the RockOn model in alphabetical order:
link to code

        return RockOn.objects.filter().order_by('name')

This list is then by processed by the renderRockons function to extract the information used in the UI—such as the rock-on’s name, description, “install” button, “link to the webUI”, or status—built through the following handlebars template:
link to code

Note that the status of all rock-ons with pending operations (install, uninstall) is updated automatically every 15 sec:
link to code

        this.updateFreq = 15000;
1 Like

Note: This is part 2 of the document, directly following the above post.

Rock-on install process

Main file of interest: rockons.js

Clicking the “Install” button for a given rock-on triggers the installRockon function, which gathers the RockOn object from the list of available rock-on (RockOnCollection fetched above) by matching the id from the install button with the id in the RockOnCollection. It then opens a new install wizard overlay by creating a new instance of the RockonInstallWizardView with the RockOn object just gathered.
link to code

    installRockon: function(event) {
        var _this = this;
        this.defTab = 0;
        event.preventDefault();
        var button = $(event.currentTarget);
        var rockon_id = button.attr('data-name');
        var rockon_o = _this.rockons.get(rockon_id);
        var wizardView = new RockonInstallWizardView({
            model: new Backbone.Model({
                rockon: rockon_o
            }),
            title: rockon_o.get('name') + ' install wizard',
            parent: this
        });
        $('.overlay-content', '#install-rockon-overlay').html(wizardView.render().el);
        $('#install-rockon-overlay').overlay().load();
    },

While we will go into the detailed logic behind the rock-on install wizard below, the process can be separated into the following steps:

  1. Rockstor first fetches all information that needs to be surfaced to the user (“which share to use for volume x?”, or “what should be the value of the environment variable y?”, or “which port to use for z?”).
  2. As the rock-on install wizard is a simple “Click next, click next, …, and submit” process, a page is created for each of the different types of information that needs customization for the given rock-on (volume, port, environment variable, …). By clicking Next or Back, the user can then navigate through these different pages and easily fill all the settings required.
  3. When the user navigate through each page, the corresponding information (filled by the user) is saved into the Backbone model.
  4. The final model used to trigger an API call to the back-end process that will first update the database models with the information provided by the user, and then begin the creation and start of the docker container(s) accordingly.

Front-end processing

As the RockonInstallWizardView is triggered with the RockOn object, Rockstor can simply fetch from it all the information to be surfaced to the user. As the settings that can be defined by the user can be of the type volume, port, custom_config, device, or environment, the RockonInstallWizardView is initialized to fetch all this different information:
link to code

    initialize: function() {
        WizardView.prototype.initialize.apply(this, arguments);
        this.pages = [];
        this.rockon = this.model.get('rockon');
        this.volumes = new RockOnVolumeCollection(null, {
            rid: this.rockon.id
        });
        this.ports = new RockOnPortCollection(null, {
            rid: this.rockon.id
        });
        this.custom_config = new RockOnCustomConfigCollection(null, {
            rid: this.rockon.id
        });
        this.devices = new RockOnDeviceCollection(null, {
            rid: this.rockon.id
        });
        this.environment = new RockOnEnvironmentCollection(null, {
            rid: this.rockon.id
        });
    },

Rockstor can now try to fetch the information for all these collections from the corresponding django models:
link to code

    fetchVolumes: function() {
        (...)
        this.volumes.fetch({
            success: function() {
                _this.model.set('volumes', _this.volumes);
                _this.fetchPorts();
            }
        });
    },

    fetchPorts: function() {
        (...)
        this.ports.fetch({
            success: function() {
                _this.model.set('ports', _this.ports);
                _this.fetchDevices();
            }
        });
    },

    fetchCustomConfig: function() {
        (...)
        this.custom_config.fetch({
            success: function() {
                _this.model.set('custom_config', _this.custom_config);
                _this.fetchEnvironment();
            }
        });
    },

    fetchDevices: function() {
        (...)
        this.devices.fetch({
            success: function() {
                _this.model.set('devices', _this.devices);
                _this.fetchCustomConfig();
            }
        });
    },

    fetchEnvironment: function() {
        (...)
        this.environment.fetch({
            success: function() {
                _this.model.set('environment', _this.environment);
                _this.addPages();
            }
        });
    },

    render: function() {
        this.fetchVolumes();
        return this;
    },

Now that the required information to be customized by the user has been fetched, Rockstor can now proceed with the creation of the different pages needed. Notably, only pages relevant to the rock-on in question are created:
link to code

    addPages: function() {
        if (this.volumes.length > 0) {
            this.pages.push(RockonShareChoice);
        }

In the snippet above, for instance, the page asking the user to select which share to use for the different volumes defined in the rock-on (RockonShareChoice) will be added to the navigation only if the rock-on actually needs this information.
There is thus one page per type of information, each rendered differently to better suit the nature of the information to be gathered (a drop-down select field for choosing shares, for instance, but text fields for environment variables), and using dedicated Handlebars templates. See the table below for more details.

Information type Function name Template name (link)
Volume RockonShareChoice vol_form.jst
Port RockonPortChoice ports_form.jst
Device RockonDeviceChoice device_form.jst
Environment variable RockonEnvironment cc_form.jst
custom_config RockonCustomChoice cc_form.jst

Let’s look at the RockonShareChoice as an illustration:
link to code

RockonShareChoice = RockstorWizardPage.extend({
    initialize: function() {
        (...)
        this.rockon = this.model.get('rockon');
        this.volumes = this.model.get('volumes');
        this.shares = new ShareCollection();
        (...)
    },

As we can see above, RockonShareChoice is initialized to fetch both (i) the volume(s) associated to this rock-on, and (ii) the available Rockstor shares. Rockstor can then create the Share choice form accordingly:
link to code

    render: function() {
        (...)
        this.shares.fetch();
        (...)
    },


    renderVolumes: function() {
        this.$('#ph-vols-table').html(this.vol_template({
            volumes: this.volumes.toJSON(),
            shares: this.shares.toJSON()
        }));
        //form validation
        (...)
    },

Now that the form has been created, we can proceed with collecting the information filled by the user (shares in this case) and saving it in the Backbone model so that we can re-use it at the end of the install wizard:
link to code

    save: function() {
        // Validate the form
        if (!this.volForm.valid()) {
            this.validator.showErrors();
            return $.Deferred().reject();
        }

        var share_map = {};
        var volumes = this.volumes.filter(function(volume) {
            share_map[this.$('#' + volume.id).val()] = volume.get('dest_dir');
            return volume;
        }, this);
        this.model.set('share_map', share_map);
        return $.Deferred().resolve();
    }
});

As seen above, if the form is valid, we create a new variable share_map in which all volume:share mappings are saved in the form share:container_volume. This share_map variable is then saved to the Backbone model.

By repeating this process for each information type needed by the given rock-on, we thus end up with the Backbone model (accessible as this.model) containing all the necessary information.

Front-end finalization

Before finalizing the install wizard, all the collected information is summarized and presented to the user for a final review. This is done in the RockonInstallSummary page:
link to code

RockonInstallSummary = RockstorWizardPage.extend({
    initialize: function() {
        this.template = window.JST.rockons_install_summary;
        this.table_template = window.JST.rockons_summary_table;
        this.share_map = this.model.get('share_map');
        this.port_map = this.model.get('port_map');
        this.cc_map = this.model.get('cc_map');
        this.dev_map = this.model.get('dev_map');
        this.env_map = this.model.get('env_map');
        this.ports = this.model.get('ports');
        this.devices = this.model.get('devices');
        this.environment = this.model.get('environment');
        this.cc = this.model.get('custom_config');
        this.rockon = this.model.get('rockon');
        RockstorWizardPage.prototype.initialize.apply(this, arguments);
    },

As detailed above, all information collected previously is stored in the Backbone model. We can thus easily get all the information we want to display to the user from this.model and present them in a table:
link to code

    render: function() {
        RockstorWizardPage.prototype.render.apply(this, arguments);
        this.$('#ph-summary-table').html(this.table_template({
            share_map: this.share_map,
            port_map: this.port_map,
            cc_map: this.cc_map,
            dev_map: this.dev_map,
            env_map: this.env_map
        }));
        return this;
    },

Now that the user has all the information at her/his disposal, (s)he can submit the form and send the relevant data to the corresponding API:
link to code

    save: function() {
        (...)
        return $.ajax({
            url: '/api/rockons/' + this.rockon.id + '/install',
            type: 'POST',
            dataType: 'json',
            contentType: 'application/json',
            data: JSON.stringify({
                'ports': this.port_map,
                'shares': this.share_map,
                'cc': this.cc_map,
                'devices': this.dev_map,
                'environment': this.env_map
            }),
            (...)

As we can see above, by clicking the “Submit” button, a API request of type POST is sent to the corresponding url with the rock-on ID (this.rockon.id) and the install command. Notably, this request is sent with all the information filled by the user (and saved into the Backbone model).

Back-end processing

Main file of interest: rockon_id.py

As all the required information is present in the POST request, we simply need to parse it and update the relevant database models accordingly:
link to code

    def post(self, request, rid, command):
        (...)
            try:
                rockon = RockOn.objects.get(id=rid)
            (...)

            if (command == 'install'):
                (...)
                share_map = request.data.get('shares', {})
                port_map = request.data.get('ports', {})
                dev_map = request.data.get('devices', {})
                cc_map = request.data.get('cc', {})
                env_map = request.data.get('environment', {})
                containers = DContainer.objects.filter(rockon=rockon)
                for co in containers:
                    for sname in share_map.keys():
                        # Deal with volume / shares
                    
                    for p in port_map.keys():
                        # Deal with ports
                    for d in dev_map.keys():
                        # Deal with devices
                    for c in cc_map.keys():
                        # Deal with custom_config
                    for e in env_map.keys():
                        # Deal with environment variables
                install.async(rockon.id)
                rockon.state = 'pending_install'
                rockon.save()

As summarized above, the overall logic is fairly simple. First, we get the RockOn object from the id contained in the API call (rid), as well as its underlying container(s) from the DContainer model, and the information provided by the user in the install wizard (share_map, etc…). We can then proceed with updating the different models associated to each container of the given rock-on with the relevant information. Once all models of interest have been updated, we can finally begin the lower level steps of the installation process: building the docker commands (triggered by install.async(rockon.id)).

Building the docker command

Main file of interest: rockon_helpers.py

Now that all the information has been updated in the database, we are finally ready to build and run the docker command: docker run.

The overall logic is, again, fairly simple but somewhat extensive due to the variety of options and settings that can be added to the docker run command. To simplify the process, Rockstor has separate helper functions for each type of information (volume, ports, etc…).

Notably, as two rock-ons (owncloud, and openvpn) have dedicated install procedures for historic reasons, we first need to trigger the right procedure: either specific to these two rock-ons, or a generic one. This is accomplished as follows:
link to code

def install(rid):
    (...)
    try:
        rockon = RockOn.objects.get(id=rid)
        globals().get('%s_install' % rockon.name.lower(),
                      generic_install)(rockon)
        (...)

As the generic_install process is valid for all rock-ons except the two listed above, we will now focus on this procedure.
link to code

def generic_install(rockon):
    for c in DContainer.objects.filter(rockon=rockon).order_by('launch_order'):
        rm_container(c.name)
        # pull image explicitly so we get updates on re-installs.
        image_name_plus_tag = c.dimage.name + ':' + c.dimage.tag
        run_command([DOCKER, 'pull', image_name_plus_tag], log=True)
        cmd = list(DCMD2) + ['--name', c.name, ]
        cmd.extend(vol_ops(c))
        # Add '--device' flag
        cmd.extend(device_ops(c))
        (...)
        cmd.extend(port_ops(c))
        cmd.extend(container_ops(c))
        cmd.extend(envars(c))
        cmd.extend(labels_ops(c))
        cmd.append(image_name_plus_tag)
        cmd.extend(cargs(c))
        run_command(cmd, log=True)

This procedure is, again, fairly simple, and can be broken down to the following important points:

  1. We loop through all containers included in the rock-on.
  2. We attempt to remove any already existing container with the same name.
  3. We pull the container image. Following docker’s default behavior, the image will be pulled only if a newer image exists on the repository. Note how we are also pulling specifically the image with the tag specified in the JSON file (or latest otherwise).
  4. We then create the base docker run command (DCMD2) with the name of the container as specified in the JSON file.
  5. We then complete the docker run command by trying to add all additional settings (volumes, ports, arguments, etc…) using dedicated helper functions such as vol_ops(), for instance. See below for more details.
  6. Once the command is completed, we can simply run it (run_command(cmd, log=True).

All helper functions listed in step 5 above (see table below) follow the same overall logic: gather all corresponding information from the database and build the appropriate docker run option. Defining environment variables, for instance, is taken care of by envars(c)–note how this helper function runs at the container level:
link to code

def envars(container):
    var_list = []
    for e in DContainerEnv.objects.filter(container=container):
        var_list.extend(['-e', '%s=%s' % (e.key, e.val)])
    return var_list

As you can see above, we build a string starting with the corresponding docker run flag -e followed by the environment_variable_name:value combination. We thus end up with a properly formatted list of environment_variable options to add to the docker run command.

Note also how the docker run command is used with the --restart=unless-stopped option:
link to code

DCMD2 = list(DCMD) + ['-d', '--restart=unless-stopped', ]
Information type Helper name link
Option container_ops() link to code
Port ports_ops() link to code
Volume vol_ops() link to code
Device device_ops() link to code
Command argument cargs_ops() link to code
Container label labels_ops() link to code
Environment variable envars() link to code

Rock-on settings post install customization

Some settings can be further customized beyond what is offered and possible during the installation wizard. Although only a few options are supported at the moment, this area is under active development and thus likely to be substantially improved upon. As of this writing, the following is supported: Add storage, and Add label.

Both customizations follow the same overall logic:

  1. Create a dedicated form in the webUI allowing the user to easily specify new settings of interest (add a new volume, and/or add labels to the containers).
  2. Update the relevant database models with the new information provided by the user.
  3. Uninstall the current docker container(s) underlying the rock-on, and rebuild them. As all containers are built using information from the database at the time of building, the newly-built containers will include the new settings provided by the user in step 1 above.

The front-end part of the process is very similar to the rock-on install wizard and will thus not be covered here. Nevertheless, the finalization of the front-end process differs notably in its API call. Indeed, while the rock-on install wizard sends a request with the update command (see above), the rock-on update process sends the update command instead. As a result, the way the request data is processed differs greatly. We will now go over some details below.
link to code

    save: function() {
        (...)
        return $.ajax({
            url: '/api/rockons/' + this.rockon.id + '/update',
            type: 'POST',
            dataType: 'json',
            contentType: 'application/json',
            data: JSON.stringify({
                'shares': this.shares,
                'labels': this.new_labels
            }),
            success: function() {}
        });
    }

Main file of interest: rockon_id.py
link to code

            elif (command == 'update'):
                (...)
                share_map = request.data.get('shares')
                label_map = request.data.get('labels')
                if bool(share_map):
                    for co in DContainer.objects.filter(rockon=rockon):
                        for s in share_map.keys():
                            (...) # Checks and verifications
                            do = DVolume(container=co, share=so, uservol=True,
                                         dest_dir=s)
                            do.save()
                if bool(label_map):
                    for co in DContainer.objects.filter(rockon=rockon):
                        for c in label_map.keys():
                            cname = label_map[c]
                            coname = co.name
                            if(cname != coname):
                                continue
                            lo = DContainerLabel(container=co, key=cname, val=c)
                            lo.save()
                rockon.state = 'pending_update'
                rockon.save()
                update.async(rockon.id)

As during the rock-on install process, we get the needed information from the request data and update the relevant database models accordingly. Then, we can begin the lower level steps of the update process triggered here with the update.async(rockon.id) call.

Main file of interest: rockon_helpers.py
link to code

def update(rid):
    uninstall(rid, new_state='pending_update')
    install(rid)

The process here is very simple as we simply need to trigger the current rock-on uninstallation, followed by its re-installation.

Rock-on start/stop

Main file of interest: rockons.js
Starting and stopping a rock-on is controlled by the rockonToggle function which simply sends a request to the rock-on API with either the start or stop command:
link to code

    rockonToggle: function(event, state) {
        var rockonId = $(event.target).attr('data-rockon-id');
        if (state) {
            this.startRockon(rockonId);
        } else {
            this.stopRockon(rockonId);
        }
    },

    startRockon: function(rockonId) {
        (...)
        $.ajax({
            url: '/api/rockons/' + rockonId + '/start',
            type: 'POST',
            dataType: 'json',
            success: function(data, status, xhr) {
                _this.defTab = 0;
                _this.updateStatus();
            },
            (...)
        });
    },

    stopRockon: function(rockonId) {
        (...)
        $.ajax({
            url: '/api/rockons/' + rockonId + '/stop',
            type: 'POST',
            dataType: 'json',
            success: function(data, status, xhr) {
                _this.defTab = 0;
                _this.updateStatus();
            },
            (...)
        });
    },

The request is then processed accordingly by rockon_id.py depending on the command received (start or stop):
link to code

            elif (command == 'stop'):
                stop.async(rockon.id)
                rockon.status = 'pending_stop'
                rockon.save()
            elif (command == 'start'):
                start.async(rockon.id)
                rockon.status = 'pending_start'
                rockon.save()

As during the install and update processes, the stop() and start() commands are simple wrappers to corresponding docker commands:
link to code

Rock-on uninstall

Main file of interest: rockons.js
Uninstalling a rock-on is controlled by the uninstallRockon function which simply sends a request to the rock-on API with either the uninstall command:
link to code

    uninstallRockon: function(event) {
        (...)
            $.ajax({
                url: '/api/rockons/' + rockon_id + '/uninstall',
                type: 'POST',
                dataType: 'json',
                (...)
            });
        (...)

The request is then processed accordingly by rockon_id.py:
link to code

            elif (command == 'uninstall'):
                (...)
                uninstall.async(rockon.id)
                rockon.state = 'pending_uninstall'
                rockon.save()
                for co in DContainer.objects.filter(rockon=rockon):
                    DVolume.objects.filter(container=co, uservol=True).delete()
                    DContainerLabel.objects.filter(container=co).delete()

Note how in addition to starting the lower level uninstall() procedure (see below), we also need to delete instances in the database that were eventually added as post-install customization for this rock-on:

  • DVolume instances with the uservol flag set to True (created with the Add Storage feature).
  • DContainerLabel instances (created with the Add Label feature).

As during the install and update processes, the uninstall() command is a simple wrapper to corresponding docker command:
link to code

def uninstall(rid, new_state='available'):
    try:
        rockon = RockOn.objects.get(id=rid)
        globals().get('%s_uninstall' % rockon.name.lower(),
                      generic_uninstall)(rockon)
    (...)

def generic_uninstall(rockon):
    for c in DContainer.objects.filter(rockon=rockon):
        rm_container(c.name)

def rm_container(name):
    o, e, rc = run_command([DOCKER, 'stop', name], throw=False, log=True)
    o, e, rc = run_command([DOCKER, 'rm', name], throw=False, log=True)
    (...)

As you can see above, uninstalling a rock-on removes only the underlying container(s) but does not delete the corresponding docker image(s). The latter still reside in the share selected as the rockons-root during the rock-on service setup (see Rockstor documentation). While this allows a quick re-installation later on, it can also lead to an accumulation of unused images on the system. There are plans, however, to limit this issue by offering a better degree of management for docker images from Rockstor’s webUI in the near future.

Future evolution

The reader is referred to another wiki post (linked below) dedicated to the discussion of the evolution and improvements of the rock-on framework:

1 Like

Thank you for your documentation effort. TBH, since i have portainer running (privileged via Unix socket) as the mgmt UI i’ve stopped looking for further replacements. Yes, its that good & easy to use.

I would wish the Rock-on UI could be similar.