Once again Rockstor web interface does not start.
Previous times I had the same problem it was because nginx.conf file was empty.
So I ran the commands to re-populate it as described here.
But this time the problem remains.
Please help me troubleshoot. My SMB shares are all accessible.
It’s the web interface that just won’t load.
This is the output of systemctl status -l roskctor:
Nov 28 15:53:13 nas supervisord[8445]: 2020-11-28 15:53:13,150 INFO spawned: ‘nginx’ with pid 8521
Nov 28 15:53:13 nas supervisord[8445]: 2020-11-28 15:53:13,157 INFO exited: nginx (exit status 127; not expected)
Nov 28 15:53:14 nas supervisord[8445]: 2020-11-28 15:53:14,159 INFO success: data-collector entered RUNNING state, process has stayed up for > than 2 seconds (startsecs)
Nov 28 15:53:14 nas supervisord[8445]: 2020-11-28 15:53:14,159 INFO success: ztask-daemon entered RUNNING state, process has stayed up for > than 2 seconds (startsecs)
Nov 28 15:53:15 nas supervisord[8445]: 2020-11-28 15:53:15,162 INFO spawned: ‘nginx’ with pid 8732
Nov 28 15:53:15 nas supervisord[8445]: 2020-11-28 15:53:15,169 INFO exited: nginx (exit status 127; not expected)
Nov 28 15:53:17 nas supervisord[8445]: 2020-11-28 15:53:17,172 INFO success: gunicorn entered RUNNING state, process has stayed up for > than 5 seconds (startsecs)
Nov 28 15:53:18 nas supervisord[8445]: 2020-11-28 15:53:18,176 INFO spawned: ‘nginx’ with pid 8755
Nov 28 15:53:18 nas supervisord[8445]: 2020-11-28 15:53:18,183 INFO exited: nginx (exit status 127; not expected)
Nov 28 15:53:19 nas supervisord[8445]: 2020-11-28 15:53:19,185 INFO gave up: nginx entered FATAL state, too many start retries too quickly
That is a curious one. It seems you are on the right track with Nginx and I’m particularly interested in the following error you shared:
To the best of my knowledge, an exit status of 127 refers to a command not found, which may seem supervisord has a problem with starting the nginx binary. If we look at its configuration, we can see the following:
# cat /opt/rockstor/etc/supervisord.conf
(...)
; Programs
; Nginx
[program:nginx]
command=/usr/sbin/nginx -c /opt/rockstor/etc/nginx/nginx.conf ; the program (relative uses PATH, can take args)
(...)
The question now is whether or not something happened to your nginx installation. Let’s try to verify it’s all as it should be:
Is nginx installed?
# yum info nginx
Installed Packages
Name : nginx
Version : 1.16.1
Release : lp152.1.3
Architecture : x86_64
Size : 2.6 M
Source : nginx-1.16.1-lp152.1.3.src.rpm
Repository : @System
Summary : A HTTP server and IMAP/POP3 proxy server
URL : https://nginx.org/
License : BSD-2-Clause
Description : nginx [engine x] is a HTTP server and IMAP/POP3 proxy server written by Igor Sysoev.
: It has been running on many heavily loaded Russian sites for more than two years.
If yes, can the binary be found where it should be?
# which nginx
/usr/sbin/nginx
Let’s see where these lead us.
You didn’t specify whether you made any change to your system before you noticed something wrong with it. It may be some system updates that didn’t finish correctly, or something along those lines. Regardless, let us know what these two commands above have to say, so we can see whether something is wrong with nginx on your side.
No changes in my Rockstor system. The only difference is I added a mediaportal tv server which tried (unsuccessfully) to write a recorded file to a samba shared folder located in Rockstor NAS.
It seems I am facing a permissions’ issue there.
And now to the commands,
yum info nginx
Loaded plugins: changelog, fastestmirror
Loading mirror speeds from cached hostfile
updates: ftp.cc.uoc.gr
Installed Packages
Name : nginx
Arch : x86_64
Epoch : 1
Version : 1.16.1
Release : 3.el7
Size : 1.6 M
Repo : installed
From repo : epel
Summary : A high performance web server and reverse proxy server
URL : http://nginx.org/
License : BSD
Description : Nginx is a web server and a reverse proxy server for HTTP, SMTP, POP3 and
: IMAP protocols, with a strong focus on high concurrency, performance and low
: memory usage.
It does, indeed. Let’s try to see what supervisord has to say in this case: /opt/build/bin/supervisorctl status nginx
Mine shows:
opt/rockstor/bin/supervisorctl status nginx
nginx RUNNING pid 1148, uptime 0:19:26
if not running (which it seems to not be the case for you), then, you can monitor the logs when you try to start it. You can open two different terminal windows:
terminal session 1: monitor the logs: tail -f /opt/rockstor/var/log/supervisord_nginx_std*
/opt/rockstor/bin/supervisorctl status nginx
/opt/rockstor/eggs/setuptools-46.1.3-py2.7.egg/pkg_resources/py2_warn.py:21: UserWarning: Setuptools will stop working on Python 2
You are running Setuptools on Python 2, which is no longer
supported and
SETUPTOOLS WILL STOP WORKING <<<
in a subsequent release (no sooner than 2020-04-20).
Please ensure you are installing
Setuptools using pip 9.x or later or pin to setuptools<45
in your environment.
If you have done those things and are still encountering
this message, please follow up at https://bit.ly/setuptools-py2-warning.
sys.version_info < (3,) and warnings.warn(pre + “" * 60 + msg + "” * 60)
nginx FATAL Exited too quickly (process log may have details)
And thereafter I have:
/opt/rockstor/bin/supervisorctl start nginx
/opt/rockstor/eggs/setuptools-46.1.3-py2.7.egg/pkg_resources/py2_warn.py:21: UserWarning: Setuptools will stop working on Python 2
You are running Setuptools on Python 2, which is no longer
supported and
SETUPTOOLS WILL STOP WORKING <<<
in a subsequent release (no sooner than 2020-04-20).
Please ensure you are installing
Setuptools using pip 9.x or later or pin to setuptools<45
in your environment.
If you have done those things and are still encountering
this message, please follow up at https://bit.ly/setuptools-py2-warning.
tail -f /opt/rockstor/var/log/supervisord_nginx_std*
==> /opt/rockstor/var/log/supervisord_nginx_stderr.log <==
/usr/sbin/nginx: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory
/usr/sbin/nginx: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory
/usr/sbin/nginx: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory
/usr/sbin/nginx: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory
/usr/sbin/nginx: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory
/usr/sbin/nginx: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory
/usr/sbin/nginx: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory
/usr/sbin/nginx: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory
/usr/sbin/nginx: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory
/usr/sbin/nginx: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory
==> /opt/rockstor/var/log/supervisord_nginx_stderr.log <==
/usr/sbin/nginx: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory
/usr/sbin/nginx: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory
/usr/sbin/nginx: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory
/usr/sbin/nginx: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory
Most likely the cause of your problems . If I’m correct (I’m doing this from memory at the moment), the libssl libraries are provided by the openssl package, so let’s see how it is in your system. I believe you are running Rockstor 3.9.x, (CentOS) isn’t it? If that’s correct, the following commands should help: yum info openssl
and then try to reinstall it: yum reinstall openssl
I had the same problem recently. The version of openssl-libs installed is 1.0 and the newer nginx wants 1.1 of the libs.
Install openssl11-libs but not openssl11 as that could create problems with other dependencies. Restart after the install and see if the Rockstor UI comes up.
Mine has been running for a few days since without problems.
I will (yum) install openssl11-libs as soon as I can, because at the moment my machine is in use.
I will get back to you about this.
How did nginx got updated (v1.1)? Was it a Rockstor automatic update? I certainly did not do the update manually.
For me it was the manual periodic updates I run. It looks like CentOS has been releasing again after a quiet period (el8?) and nginx updated for me on 11/26 when I noticed the UI problem.
The strange thing is if I run a dependency check for the nginx package it does list openssl11-libs so in my view it should have installed that during the update. This check is after the fact so I don’t know how it looked before installing the new libs.
Strange one, indeed, but thanks for sharing the solution with us! To the best of my knowledge, we are indeed relying on upstream (epel repo) for these packages so it’s a bit odd to see such a dependency issue.
@Sky12016, thanks a lot for confirming this fixed the issue on your system as well… I’m glad you’re now all set!
Hi, I just checked my install history, and in my case (on Centos) the openssl11-libs were installed as a dependency of nginx successfully - only difference for me is that I did not use the command line to initiate the updates but probably clicked on the update icon in the WebUi at the time (11/23). A subsequent update apparently came then on 11/29 (not initiated by me at all but part of the auto-update cycle, I guess).
In my case, I never update manually and every time (not frequently) I log into Rockstor’s web-gui there are no updates to be installed. So apparently the issue was created from the automatic (stable) updates’ channel.
One of my favourite things about Rockstor is the set-it-and-forget-it approach which just works!
. If I’m correct (I’m doing this from memory at the moment), the libssl libraries are provided by the 
