Note: This is part 2 of the document, directly following the above post.
Rock-on install process
Main file of interest: rockons.js
Clicking the “Install” button for a given rock-on triggers the installRockon function, which gathers the RockOn object from the list of available rock-on (RockOnCollection fetched above) by matching the id from the install button with the id in the RockOnCollection. It then opens a new install wizard overlay by creating a new instance of the RockonInstallWizardView with the RockOn object just gathered.
link to code
installRockon: function(event) {
var _this = this;
this.defTab = 0;
event.preventDefault();
var button = $(event.currentTarget);
var rockon_id = button.attr('data-name');
var rockon_o = _this.rockons.get(rockon_id);
var wizardView = new RockonInstallWizardView({
model: new Backbone.Model({
rockon: rockon_o
}),
title: rockon_o.get('name') + ' install wizard',
parent: this
});
$('.overlay-content', '#install-rockon-overlay').html(wizardView.render().el);
$('#install-rockon-overlay').overlay().load();
},
While we will go into the detailed logic behind the rock-on install wizard below, the process can be separated into the following steps:
- Rockstor first fetches all information that needs to be surfaced to the user (“which share to use for volume x?”, or “what should be the value of the environment variable y?”, or “which port to use for z?”).
- As the rock-on install wizard is a simple “Click next, click next, …, and submit” process, a page is created for each of the different types of information that needs customization for the given rock-on (volume, port, environment variable, …). By clicking Next or Back, the user can then navigate through these different pages and easily fill all the settings required.
- When the user navigate through each page, the corresponding information (filled by the user) is saved into the Backbone model.
- The final model used to trigger an API call to the back-end process that will first update the database models with the information provided by the user, and then begin the creation and start of the docker container(s) accordingly.
Front-end processing
As the RockonInstallWizardView is triggered with the RockOn object, Rockstor can simply fetch from it all the information to be surfaced to the user. As the settings that can be defined by the user can be of the type volume, port, custom_config, device, or environment, the RockonInstallWizardView is initialized to fetch all this different information:
link to code
initialize: function() {
WizardView.prototype.initialize.apply(this, arguments);
this.pages = [];
this.rockon = this.model.get('rockon');
this.volumes = new RockOnVolumeCollection(null, {
rid: this.rockon.id
});
this.ports = new RockOnPortCollection(null, {
rid: this.rockon.id
});
this.custom_config = new RockOnCustomConfigCollection(null, {
rid: this.rockon.id
});
this.devices = new RockOnDeviceCollection(null, {
rid: this.rockon.id
});
this.environment = new RockOnEnvironmentCollection(null, {
rid: this.rockon.id
});
},
Rockstor can now try to fetch the information for all these collections from the corresponding django models:
link to code
fetchVolumes: function() {
(...)
this.volumes.fetch({
success: function() {
_this.model.set('volumes', _this.volumes);
_this.fetchPorts();
}
});
},
fetchPorts: function() {
(...)
this.ports.fetch({
success: function() {
_this.model.set('ports', _this.ports);
_this.fetchDevices();
}
});
},
fetchCustomConfig: function() {
(...)
this.custom_config.fetch({
success: function() {
_this.model.set('custom_config', _this.custom_config);
_this.fetchEnvironment();
}
});
},
fetchDevices: function() {
(...)
this.devices.fetch({
success: function() {
_this.model.set('devices', _this.devices);
_this.fetchCustomConfig();
}
});
},
fetchEnvironment: function() {
(...)
this.environment.fetch({
success: function() {
_this.model.set('environment', _this.environment);
_this.addPages();
}
});
},
render: function() {
this.fetchVolumes();
return this;
},
Now that the required information to be customized by the user has been fetched, Rockstor can now proceed with the creation of the different pages needed. Notably, only pages relevant to the rock-on in question are created:
link to code
addPages: function() {
if (this.volumes.length > 0) {
this.pages.push(RockonShareChoice);
}
In the snippet above, for instance, the page asking the user to select which share to use for the different volumes defined in the rock-on (RockonShareChoice) will be added to the navigation only if the rock-on actually needs this information.
There is thus one page per type of information, each rendered differently to better suit the nature of the information to be gathered (a drop-down select field for choosing shares, for instance, but text fields for environment variables), and using dedicated Handlebars templates. See the table below for more details.
| Information type | Function name | Template name (link) |
|---|---|---|
| Volume | RockonShareChoice | vol_form.jst |
| Port | RockonPortChoice | ports_form.jst |
| Device | RockonDeviceChoice | device_form.jst |
| Environment variable | RockonEnvironment | cc_form.jst |
| custom_config | RockonCustomChoice | cc_form.jst |
Let’s look at the RockonShareChoice as an illustration:
link to code
RockonShareChoice = RockstorWizardPage.extend({
initialize: function() {
(...)
this.rockon = this.model.get('rockon');
this.volumes = this.model.get('volumes');
this.shares = new ShareCollection();
(...)
},
As we can see above, RockonShareChoice is initialized to fetch both (i) the volume(s) associated to this rock-on, and (ii) the available Rockstor shares. Rockstor can then create the Share choice form accordingly:
link to code
render: function() {
(...)
this.shares.fetch();
(...)
},
renderVolumes: function() {
this.$('#ph-vols-table').html(this.vol_template({
volumes: this.volumes.toJSON(),
shares: this.shares.toJSON()
}));
//form validation
(...)
},
Now that the form has been created, we can proceed with collecting the information filled by the user (shares in this case) and saving it in the Backbone model so that we can re-use it at the end of the install wizard:
link to code
save: function() {
// Validate the form
if (!this.volForm.valid()) {
this.validator.showErrors();
return $.Deferred().reject();
}
var share_map = {};
var volumes = this.volumes.filter(function(volume) {
share_map[this.$('#' + volume.id).val()] = volume.get('dest_dir');
return volume;
}, this);
this.model.set('share_map', share_map);
return $.Deferred().resolve();
}
});
As seen above, if the form is valid, we create a new variable share_map in which all volume:share mappings are saved in the form share:container_volume. This share_map variable is then saved to the Backbone model.
By repeating this process for each information type needed by the given rock-on, we thus end up with the Backbone model (accessible as this.model) containing all the necessary information.
Front-end finalization
Before finalizing the install wizard, all the collected information is summarized and presented to the user for a final review. This is done in the RockonInstallSummary page:
link to code
RockonInstallSummary = RockstorWizardPage.extend({
initialize: function() {
this.template = window.JST.rockons_install_summary;
this.table_template = window.JST.rockons_summary_table;
this.share_map = this.model.get('share_map');
this.port_map = this.model.get('port_map');
this.cc_map = this.model.get('cc_map');
this.dev_map = this.model.get('dev_map');
this.env_map = this.model.get('env_map');
this.ports = this.model.get('ports');
this.devices = this.model.get('devices');
this.environment = this.model.get('environment');
this.cc = this.model.get('custom_config');
this.rockon = this.model.get('rockon');
RockstorWizardPage.prototype.initialize.apply(this, arguments);
},
As detailed above, all information collected previously is stored in the Backbone model. We can thus easily get all the information we want to display to the user from this.model and present them in a table:
link to code
render: function() {
RockstorWizardPage.prototype.render.apply(this, arguments);
this.$('#ph-summary-table').html(this.table_template({
share_map: this.share_map,
port_map: this.port_map,
cc_map: this.cc_map,
dev_map: this.dev_map,
env_map: this.env_map
}));
return this;
},
Now that the user has all the information at her/his disposal, (s)he can submit the form and send the relevant data to the corresponding API:
link to code
save: function() {
(...)
return $.ajax({
url: '/api/rockons/' + this.rockon.id + '/install',
type: 'POST',
dataType: 'json',
contentType: 'application/json',
data: JSON.stringify({
'ports': this.port_map,
'shares': this.share_map,
'cc': this.cc_map,
'devices': this.dev_map,
'environment': this.env_map
}),
(...)
As we can see above, by clicking the “Submit” button, a API request of type POST is sent to the corresponding url with the rock-on ID (this.rockon.id) and the install command. Notably, this request is sent with all the information filled by the user (and saved into the Backbone model).
Back-end processing
Main file of interest: rockon_id.py
As all the required information is present in the POST request, we simply need to parse it and update the relevant database models accordingly:
link to code
def post(self, request, rid, command):
(...)
try:
rockon = RockOn.objects.get(id=rid)
(...)
if (command == 'install'):
(...)
share_map = request.data.get('shares', {})
port_map = request.data.get('ports', {})
dev_map = request.data.get('devices', {})
cc_map = request.data.get('cc', {})
env_map = request.data.get('environment', {})
containers = DContainer.objects.filter(rockon=rockon)
for co in containers:
for sname in share_map.keys():
# Deal with volume / shares
for p in port_map.keys():
# Deal with ports
for d in dev_map.keys():
# Deal with devices
for c in cc_map.keys():
# Deal with custom_config
for e in env_map.keys():
# Deal with environment variables
install.async(rockon.id)
rockon.state = 'pending_install'
rockon.save()
As summarized above, the overall logic is fairly simple. First, we get the RockOn object from the id contained in the API call (rid), as well as its underlying container(s) from the DContainer model, and the information provided by the user in the install wizard (share_map, etc…). We can then proceed with updating the different models associated to each container of the given rock-on with the relevant information. Once all models of interest have been updated, we can finally begin the lower level steps of the installation process: building the docker commands (triggered by install.async(rockon.id)).
Building the docker command
Main file of interest: rockon_helpers.py
Now that all the information has been updated in the database, we are finally ready to build and run the docker command: docker run.
The overall logic is, again, fairly simple but somewhat extensive due to the variety of options and settings that can be added to the docker run command. To simplify the process, Rockstor has separate helper functions for each type of information (volume, ports, etc…).
Notably, as two rock-ons (owncloud, and openvpn) have dedicated install procedures for historic reasons, we first need to trigger the right procedure: either specific to these two rock-ons, or a generic one. This is accomplished as follows:
link to code
def install(rid):
(...)
try:
rockon = RockOn.objects.get(id=rid)
globals().get('%s_install' % rockon.name.lower(),
generic_install)(rockon)
(...)
As the generic_install process is valid for all rock-ons except the two listed above, we will now focus on this procedure.
link to code
def generic_install(rockon):
for c in DContainer.objects.filter(rockon=rockon).order_by('launch_order'):
rm_container(c.name)
# pull image explicitly so we get updates on re-installs.
image_name_plus_tag = c.dimage.name + ':' + c.dimage.tag
run_command([DOCKER, 'pull', image_name_plus_tag], log=True)
cmd = list(DCMD2) + ['--name', c.name, ]
cmd.extend(vol_ops(c))
# Add '--device' flag
cmd.extend(device_ops(c))
(...)
cmd.extend(port_ops(c))
cmd.extend(container_ops(c))
cmd.extend(envars(c))
cmd.extend(labels_ops(c))
cmd.append(image_name_plus_tag)
cmd.extend(cargs(c))
run_command(cmd, log=True)
This procedure is, again, fairly simple, and can be broken down to the following important points:
- We loop through all containers included in the rock-on.
- We attempt to remove any already existing container with the same name.
- We pull the container image. Following docker’s default behavior, the image will be pulled only if a newer image exists on the repository. Note how we are also pulling specifically the image with the tag specified in the JSON file (or
latestotherwise). - We then create the base
docker runcommand (DCMD2) with the name of the container as specified in the JSON file. - We then complete the
docker runcommand by trying to add all additional settings (volumes, ports, arguments, etc…) using dedicated helper functions such asvol_ops(), for instance. See below for more details. - Once the command is completed, we can simply run it (
run_command(cmd, log=True).
All helper functions listed in step 5 above (see table below) follow the same overall logic: gather all corresponding information from the database and build the appropriate docker run option. Defining environment variables, for instance, is taken care of by envars(c)–note how this helper function runs at the container level:
link to code
def envars(container):
var_list = []
for e in DContainerEnv.objects.filter(container=container):
var_list.extend(['-e', '%s=%s' % (e.key, e.val)])
return var_list
As you can see above, we build a string starting with the corresponding docker run flag -e followed by the environment_variable_name:value combination. We thus end up with a properly formatted list of environment_variable options to add to the docker run command.
Note also how the docker run command is used with the --restart=unless-stopped option:
link to code
DCMD2 = list(DCMD) + ['-d', '--restart=unless-stopped', ]
| Information type | Helper name | link |
|---|---|---|
| Option | container_ops() |
link to code |
| Port | ports_ops() |
link to code |
| Volume | vol_ops() |
link to code |
| Device | device_ops() |
link to code |
| Command argument | cargs_ops() |
link to code |
| Container label | labels_ops() |
link to code |
| Environment variable | envars() |
link to code |
Rock-on settings post install customization
Some settings can be further customized beyond what is offered and possible during the installation wizard. Although only a few options are supported at the moment, this area is under active development and thus likely to be substantially improved upon. As of this writing, the following is supported: Add storage, and Add label.
Both customizations follow the same overall logic:
- Create a dedicated form in the webUI allowing the user to easily specify new settings of interest (add a new volume, and/or add labels to the containers).
- Update the relevant database models with the new information provided by the user.
- Uninstall the current docker container(s) underlying the rock-on, and rebuild them. As all containers are built using information from the database at the time of building, the newly-built containers will include the new settings provided by the user in step 1 above.
The front-end part of the process is very similar to the rock-on install wizard and will thus not be covered here. Nevertheless, the finalization of the front-end process differs notably in its API call. Indeed, while the rock-on install wizard sends a request with the update command (see above), the rock-on update process sends the update command instead. As a result, the way the request data is processed differs greatly. We will now go over some details below.
link to code
save: function() {
(...)
return $.ajax({
url: '/api/rockons/' + this.rockon.id + '/update',
type: 'POST',
dataType: 'json',
contentType: 'application/json',
data: JSON.stringify({
'shares': this.shares,
'labels': this.new_labels
}),
success: function() {}
});
}
Main file of interest: rockon_id.py
link to code
elif (command == 'update'):
(...)
share_map = request.data.get('shares')
label_map = request.data.get('labels')
if bool(share_map):
for co in DContainer.objects.filter(rockon=rockon):
for s in share_map.keys():
(...) # Checks and verifications
do = DVolume(container=co, share=so, uservol=True,
dest_dir=s)
do.save()
if bool(label_map):
for co in DContainer.objects.filter(rockon=rockon):
for c in label_map.keys():
cname = label_map[c]
coname = co.name
if(cname != coname):
continue
lo = DContainerLabel(container=co, key=cname, val=c)
lo.save()
rockon.state = 'pending_update'
rockon.save()
update.async(rockon.id)
As during the rock-on install process, we get the needed information from the request data and update the relevant database models accordingly. Then, we can begin the lower level steps of the update process triggered here with the update.async(rockon.id) call.
Main file of interest: rockon_helpers.py
link to code
def update(rid):
uninstall(rid, new_state='pending_update')
install(rid)
The process here is very simple as we simply need to trigger the current rock-on uninstallation, followed by its re-installation.
Rock-on start/stop
Main file of interest: rockons.js
Starting and stopping a rock-on is controlled by the rockonToggle function which simply sends a request to the rock-on API with either the start or stop command:
link to code
rockonToggle: function(event, state) {
var rockonId = $(event.target).attr('data-rockon-id');
if (state) {
this.startRockon(rockonId);
} else {
this.stopRockon(rockonId);
}
},
startRockon: function(rockonId) {
(...)
$.ajax({
url: '/api/rockons/' + rockonId + '/start',
type: 'POST',
dataType: 'json',
success: function(data, status, xhr) {
_this.defTab = 0;
_this.updateStatus();
},
(...)
});
},
stopRockon: function(rockonId) {
(...)
$.ajax({
url: '/api/rockons/' + rockonId + '/stop',
type: 'POST',
dataType: 'json',
success: function(data, status, xhr) {
_this.defTab = 0;
_this.updateStatus();
},
(...)
});
},
The request is then processed accordingly by rockon_id.py depending on the command received (start or stop):
link to code
elif (command == 'stop'):
stop.async(rockon.id)
rockon.status = 'pending_stop'
rockon.save()
elif (command == 'start'):
start.async(rockon.id)
rockon.status = 'pending_start'
rockon.save()
As during the install and update processes, the stop() and start() commands are simple wrappers to corresponding docker commands:
link to code
Rock-on uninstall
Main file of interest: rockons.js
Uninstalling a rock-on is controlled by the uninstallRockon function which simply sends a request to the rock-on API with either the uninstall command:
link to code
uninstallRockon: function(event) {
(...)
$.ajax({
url: '/api/rockons/' + rockon_id + '/uninstall',
type: 'POST',
dataType: 'json',
(...)
});
(...)
The request is then processed accordingly by rockon_id.py:
link to code
elif (command == 'uninstall'):
(...)
uninstall.async(rockon.id)
rockon.state = 'pending_uninstall'
rockon.save()
for co in DContainer.objects.filter(rockon=rockon):
DVolume.objects.filter(container=co, uservol=True).delete()
DContainerLabel.objects.filter(container=co).delete()
Note how in addition to starting the lower level uninstall() procedure (see below), we also need to delete instances in the database that were eventually added as post-install customization for this rock-on:
DVolumeinstances with theuservolflag set toTrue(created with the Add Storage feature).DContainerLabelinstances (created with the Add Label feature).
As during the install and update processes, the uninstall() command is a simple wrapper to corresponding docker command:
link to code
def uninstall(rid, new_state='available'):
try:
rockon = RockOn.objects.get(id=rid)
globals().get('%s_uninstall' % rockon.name.lower(),
generic_uninstall)(rockon)
(...)
def generic_uninstall(rockon):
for c in DContainer.objects.filter(rockon=rockon):
rm_container(c.name)
def rm_container(name):
o, e, rc = run_command([DOCKER, 'stop', name], throw=False, log=True)
o, e, rc = run_command([DOCKER, 'rm', name], throw=False, log=True)
(...)
As you can see above, uninstalling a rock-on removes only the underlying container(s) but does not delete the corresponding docker image(s). The latter still reside in the share selected as the rockons-root during the rock-on service setup (see Rockstor documentation). While this allows a quick re-installation later on, it can also lead to an accumulation of unused images on the system. There are plans, however, to limit this issue by offering a better degree of management for docker images from Rockstor’s webUI in the near future.
Future evolution
The reader is referred to another wiki post (linked below) dedicated to the discussion of the evolution and improvements of the rock-on framework: