Hi,
I’ve upgraded to 3.8.13 and rebooted. Now Rockstor WebUI won’t load (404 not found). NFS, SMB & SSH all work fine. /opt/rockstor/var/log/rockstor.log just says:
[03/May/2016 20:56:48] DEBUG [smart_manager.data_collector:403] Listening on port http://127.0.0.1:8080 and on port 10843 (flash policy server)
[03/May/2016 20:57:28] DEBUG [storageadmin.views.command:152] Bootstrap operations completed
“systemctl status -l rockstor-pre rockstor rockstor-bootstrap” says:
● rockstor-pre.service - Tasks required prior to starting Rockstor
Loaded: loaded (/etc/systemd/system/rockstor-pre.service; enabled; vendor preset: disabled)
Active: active (exited) since Tue 2016-05-03 21:25:25 BST; 15min ago
Main PID: 3594 (code=exited, status=0/SUCCESS)
CGroup: /system.slice/rockstor-pre.service
May 03 21:25:23 nas03 initrock[3594]: 2016-05-03 21:25:23,177: Updating the timezone from the system
May 03 21:25:23 nas03 initrock[3594]: 2016-05-03 21:25:23,178: system timezone = Europe/London
May 03 21:25:23 nas03 initrock[3594]: 2016-05-03 21:25:23,179: Updating sshd_config
May 03 21:25:23 nas03 initrock[3594]: 2016-05-03 21:25:23,182: sshd_config already has the updates. Leaving it unchanged.
May 03 21:25:23 nas03 initrock[3594]: 2016-05-03 21:25:23,183: Running prepdb…
May 03 21:25:23 nas03 initrock[3594]: 2016-05-03 21:25:23,967: stopping firewalld…
May 03 21:25:24 nas03 initrock[3594]: 2016-05-03 21:25:24,073: firewalld stopped and disabled
May 03 21:25:24 nas03 initrock[3594]: 2016-05-03 21:25:24,166: Registered task: fs.btrfs.start_balance
May 03 21:25:25 nas03 initrock[3594]: 2016-05-03 21:25:25,625: rockstor service looks correct. Not updating.
May 03 21:25:25 nas03 initrock[3594]: 2016-05-03 21:25:25,626: rockstor-bootstrap.service looks correct. Not updating.
● rockstor.service - RockStor startup script
Loaded: loaded (/etc/systemd/system/rockstor.service; enabled; vendor preset: enabled)
Active: active (running) since Tue 2016-05-03 21:25:25 BST; 15min ago
Main PID: 5944 (supervisord)
CGroup: /system.slice/rockstor.service
├─5944 /usr/bin/python /opt/rockstor/bin/supervisord -c /opt/rockstor/etc/supervisord.conf
├─6443 /usr/bin/python /opt/rockstor/bin/gunicorn --bind=127.0.0.1:8000 --pid=/run/gunicorn.pid --workers=2 --log-file=/opt/rockstor/var/log/gunicorn.log --pythonpath=/opt/rockstor/src/rockstor --settings=settings --timeout=120 --graceful-timeout=120 wsgi:application
├─6444 /usr/bin/python /opt/rockstor/bin/data-collector
├─6445 /usr/bin/python2.7 /opt/rockstor/bin/django ztaskd --noreload --replayfailed -f /opt/rockstor/var/log/ztask.log
├─7072 /usr/bin/python /opt/rockstor/bin/gunicorn --bind=127.0.0.1:8000 --pid=/run/gunicorn.pid --workers=2 --log-file=/opt/rockstor/var/log/gunicorn.log --pythonpath=/opt/rockstor/src/rockstor --settings=settings --timeout=120 --graceful-timeout=120 wsgi:application
└─7073 /usr/bin/python /opt/rockstor/bin/gunicorn --bind=127.0.0.1:8000 --pid=/run/gunicorn.pid --workers=2 --log-file=/opt/rockstor/var/log/gunicorn.log --pythonpath=/opt/rockstor/src/rockstor --settings=settings --timeout=120 --graceful-timeout=120 wsgi:application
May 03 21:25:35 nas03 supervisord[5944]: 2016-05-03 21:25:35,010 INFO gave up: nginx entered FATAL state, too many start retries too quickly
May 03 21:34:54 nas03 supervisord[5944]: 2016-05-03 21:34:54,926 INFO spawned: ‘nginx’ with pid 11755
May 03 21:34:54 nas03 supervisord[5944]: 2016-05-03 21:34:54,939 INFO exited: nginx (exit status 1; not expected)
May 03 21:34:55 nas03 supervisord[5944]: 2016-05-03 21:34:55,944 INFO spawned: ‘nginx’ with pid 11756
May 03 21:34:55 nas03 supervisord[5944]: 2016-05-03 21:34:55,957 INFO exited: nginx (exit status 1; not expected)
May 03 21:34:57 nas03 supervisord[5944]: 2016-05-03 21:34:57,965 INFO spawned: ‘nginx’ with pid 11778
May 03 21:34:57 nas03 supervisord[5944]: 2016-05-03 21:34:57,977 INFO exited: nginx (exit status 1; not expected)
May 03 21:35:00 nas03 supervisord[5944]: 2016-05-03 21:35:00,984 INFO spawned: ‘nginx’ with pid 11779
May 03 21:35:00 nas03 supervisord[5944]: 2016-05-03 21:35:00,996 INFO exited: nginx (exit status 1; not expected)
May 03 21:35:01 nas03 supervisord[5944]: 2016-05-03 21:35:01,998 INFO gave up: nginx entered FATAL state, too many start retries too quickly
● rockstor-bootstrap.service - Rockstor bootstrapping tasks
Loaded: loaded (/etc/systemd/system/rockstor-bootstrap.service; enabled; vendor preset: disabled)
Active: active (exited) since Tue 2016-05-03 21:26:11 BST; 14min ago
Main PID: 5946 (code=exited, status=0/SUCCESS)
CGroup: /system.slice/rockstor-bootstrap.service
May 03 21:26:11 nas03 bootstrap[5946]: BTRFS device scan complete
May 03 21:26:11 nas03 bootstrap[5946]: Exception occured while bootstrapping. This could be because rockstor.service is still starting up. will wait 2 seconds and
try again. Exception: Exception while setting access_token for url(http://127.0.0.1:8000): HTTPConnectionPool(host=‘127.0.0.1’, port=8000): Max retries exceeded with url: /o/token/ (Caused by <class ‘socket.error’>: [Errno 111] Connection refused). content: None
May 03 21:26:11 nas03 bootstrap[5946]: Bootstrapping complete
May 03 21:26:11 nas03 bootstrap[5946]: Running qgroup cleanup. /opt/rockstor/bin/qgroup-clean
May 03 21:26:11 nas03 bootstrap[5946]: Running qgroup limit maxout. /opt/rockstor/bin/qgroup-maxout-limit
So nginx doesn’t look healthy. supervisord.log looks much the same as the rockstor service output.
supervisord_replication_stderr.log is full of these:
Process ReplicaScheduler-1:
Traceback (most recent call last):
File “/usr/lib64/python2.7/multiprocessing/process.py”, line 258, in _bootstrap
self.run()
File “/opt/rockstor/src/rockstor/smart_manager/replication/listener_broker.py”, line 185, in run
frontend.bind(‘tcp://%s:%d’ % (self.listener_interface, self.listener_port))
File “zmq/backend/cython/socket.pyx”, line 487, in zmq.backend.cython.socket.Socket.bind (zmq/backend/cython/socket.c:5156)
File “zmq/backend/cython/checkrc.pxd”, line 25, in zmq.backend.cython.checkrc._check_rc (zmq/backend/cython/socket.c:7535)
raise ZMQError(errno)
ZMQError: No such device
Finally, running: /opt/rockstor/bin/initrock shows:
2016-05-03 21:34:52,976: Supported kernel(/boot/vmlinuz-4.4.5-1.el7.elrepo.x86_64) is already the default
2016-05-03 21:34:53,110: /etc/rc.d/rc.local looks correct. Not updating.
2016-05-03 21:34:53,110: Checking for flash and Running flash optimizations if appropriate.
2016-05-03 21:34:53,723: Updating the timezone from the system
2016-05-03 21:34:53,724: system timezone = Europe/London
2016-05-03 21:34:53,725: Updating sshd_config
2016-05-03 21:34:53,726: sshd_config already has the updates. Leaving it unchanged.
2016-05-03 21:34:53,726: Running prepdb…
2016-05-03 21:34:54,438: stopping firewalld…
2016-05-03 21:34:54,504: firewalld stopped and disabled
2016-05-03 21:34:54,588: Registered task: fs.btrfs.start_balance
2016-05-03 21:34:57,503: rockstor service looks correct. Not updating.
2016-05-03 21:34:57,503: rockstor-bootstrap.service looks correct. Not updating.
Any ideas?