@DrC Re:
Yes we just moved to https only on that server. And “… fix that …” would involve the past few years of Rockstor install all having to update, post moving the requests to https in the first place of course.
I’m assuming from that, that you mean ship with an up-to-date at release/build time Rock-on list. That has a number of issues associated with it in itself I’m afraid as the available Rock-on information is encoded in the local Postgresql database that backs our Django Web-UI framework.
Yes we do this with the jslibs during a build. And with some config files to see if they need updating during each boot.
Again this is difficult to reflect in the DB. I’m feeling like this is another whole level of complexity to address something that is really not a problem unless one is on a very slow connection. And the use case of no connection is mute as one can’t install anything anyway in that case. We are, in your suggestion, not starting from a clean slate each time (as per our current arrangement) but form an ever increasing number of initial states depending on which Rockstor version one installed from. I like the simplicity of always starting from scratch and then we only have to test if a clean system can update itself from that know state. I.e. no migration from prior state, at least from initial install. I appreciate that we then update from a miriade of states so their is that. But initial state is always blank and always has been so we can always return a system to that if something goes astray. Where as if we ship with a moving target of initial states that introduces more variance for little benefit. Simple often wins in the long run is what I’m thinking. And we pull down significantly less than a single typical web page view with every update. So there may not be a problem to solve here.
We do have, see the GitHub - rockstor/rockon-registry: hosted registry for Rock-ons re:
Upload the file to /opt/rockstor/rockons-metastore/[app].json. Hit update in the Web-UI and install your brand new Rock-on!
There are some good ideas here. Thanks for your thoughts. I particularly like the checksum download, however if nothing but a single Rock-on definition has changed then this checksum would have to cover the entire repository: again entirely doable and I think this is the nuget of agreement re improving things: post moving to server side compression of course. Plus older existing systems simply ignore this new checksum and continue doing the brute force method.
This is against the terms of service is my understanding. But they do now host packaging service so that may be doable. There is always value in hosting your own mechanism and in this case we do actually test, internally, against GitHub before publishing to the main server. This gives us a real world scenario to test against before we release each rock-on change. And incidnetally it’s painfully slow to update directly from GitHub. You can try this youself via code edits as we do for testing. It takes ages to retrieve all the files. I think they may rate limit that kind of access actually. Yes hosting directly from GitHub this kind of thing in not the way we are going. Unless they open a packaging service that matches our needs. Plus, as stated, there can be value in hosting ones own services. And also downsides of course :).
Yes we do this prior to publicly publishing. It’s a pretty nice tests as given it’s so slow it can double check our function in slow network environments.
All great ideas by the way. But as always the details are the killer. But definitely some stuff to discuss further here. @Flox is master of Rock-ons so he may well have input on some of these ideas also.
To know more about the mechanisms involved see @Flox’s excellent developer write-up here:
You might like that 
.
Thanks again for you input. We would definitely like to refine the system. But ideally not at the expense of complexity. Hence an initial move to a checksum to see if anything else at all needs to be pulled. In most cases this would avoid a tone of otherwise required calls/processing etc.