This isn’t as good as a built-in option, but I was also annoyed by this, so I just went and created a rockon to do it: https://github.com/rockstor/rockon-registry/pull/77
If you want to try it out before that PR gets merged, just grab the .json file and follow the instructions at https://github.com/rockstor/rockon-registry#how-can-i-add-my-own-app to install it.
Uses https://hub.docker.com/r/hope/redirect-http-to-https/ under the hood, which seems to work well enough for my needs.