System unavailable after update to stable 4.6.1-0 Release

Unfortunatley my system is unavailable after starting the update to the latest stable release.

I received an output mail from the cron job with hundreds of warnings.
e.g.

warning: file /opt/rockstor/static/storageadmin/js/views/version.js: remove failed: No such file or directory

warning: file /opt/rockstor/static/storageadmin/js/views/version.b13719baed01.js: remove failed: No such file or directory

warning: file /opt/rockstor/static/storageadmin/js/views/users.js: remove failed: No such file or directory

warning: file /opt/rockstor/static/storageadmin/js/views/users.2327f3b6f2f5.js: remove failed: No such file or directory

warning: file /opt/rockstor/static/storageadmin/js/views/update_certificate.js: remove failed: No such file or directory

warning: file /opt/rockstor/static/storageadmin/js/views/update_certificate.3abfc91595ab.js: remove failed: No such file or directory

warning: file /opt/rockstor/static/storageadmin/js/views/tasks.js: remove failed: No such file or directory

What went wrong?

Update:
trying to reach the WebUI fails with ERR_CONNECTION_REFUSED
when trying to open a samba share which was already known on my desktop, user & password are requested but not accepted.

@anborn Hello there.

We have the following from a prior post of yours:

Can you give us more info on what version you were running before the update and if you re-enabled the stable repository before this update. And how you disabled it in the first place. We have had a similar report of prior disablement causing issues but the update then worked OK once the repo was re-enabled.

Keep in mind also that we have begun a new testing channel phase: I assume you haven’t inadvertently updated from that as it is know broken and developers only.

Have you tried a command line reboot.

Hope that helps.

1 Like

Thanks for your fast reply.
I updated from 4.1 (stable) to the latest stable rockstor release 4.6.1-0. Rockstor repository was enabled again just before starting the update process via WebUI. There was no error message, but after 5 minutes the url refreshed and i got the message ERR_CONNECTION_REFUSED from the browser. I even do not get the login page.
openSUSE was already updated to Leap 15.4 a few weeks ago.

I also rebooted the server via command line but still the same issue.

It seems that rockstor is running, but access is blocked by the server using url to WebUI or access data via samba share.

2 Likes
NAS001:/ # systemctl status rockstor
○ rockstor.service - Rockstor startup script
     Loaded: loaded (/usr/lib/systemd/system/rockstor.service; disabled; vendor preset: disabled)
     Active: inactive (dead)

Jul 24 20:14:13 NAS001 systemd[1]: Dependency failed for Rockstor startup script.
Jul 24 20:14:13 NAS001 systemd[1]: rockstor.service: Job rockstor.service/start failed with result 'dependency'.
NAS001:/ # systemctl status rockstor-pre.service
× rockstor-pre.service - Tasks required prior to starting Rockstor
     Loaded: loaded (/usr/lib/systemd/system/rockstor-pre.service; disabled; vendor preset: disabled)
     Active: failed (Result: exit-code) since Mon 2023-07-24 20:14:13 CEST; 2h 38min ago
    Process: 847 ExecStart=/root/.local/bin/poetry run initrock (code=exited, status=1/FAILURE)
   Main PID: 847 (code=exited, status=1/FAILURE)

Jul 24 20:14:12 NAS001 systemd[1]: Starting Tasks required prior to starting Rockstor...
Jul 24 20:14:13 NAS001 poetry[847]: Traceback (most recent call last):
Jul 24 20:14:13 NAS001 poetry[847]:   File "<string>", line 1, in <module>
Jul 24 20:14:13 NAS001 poetry[847]:   File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module
Jul 24 20:14:13 NAS001 poetry[847]:     __import__(name)
Jul 24 20:14:13 NAS001 poetry[847]: ImportError: No module named scripts.initrock
Jul 24 20:14:13 NAS001 systemd[1]: rockstor-pre.service: Main process exited, code=exited, status=1/FAILURE
Jul 24 20:14:13 NAS001 systemd[1]: rockstor-pre.service: Failed with result 'exit-code'.
Jul 24 20:14:13 NAS001 systemd[1]: Failed to start Tasks required prior to starting Rockstor.
1 Like

for me it looks similar to the Issue
Rockstor-Pre Service failing after Update to 5.0.0-0

Hi @anborn,

It looks like Poetry install step may have failed during the install of the 4.6.1-0 rpm. Let’s see if we can see exactly why that happened.
Could you please check how your rockstor folder looks like?

ls -lah /opt/rockstor

And what does the poetry install logs include?

cat /opt/rockstor/poetry-install.txt

Hopefully that will provide us with a bit more information as to what went wrong.

1 Like
NAS001:~ # ls -lah /opt/rockstor
total 956K
drwxr-xr-x 1 root root  406 Jul 24 23:45 .
drwxr-xr-x 1 root root   36 Mar 15  2022 ..
-rw-r--r-- 1 root root    0 Apr 17  2021 .initrock
drwxr-xr-x 1 root root   76 Jul 24 23:45 .venv
drwxr-xr-x 1 root root   76 Jul 24 19:57 .venv.old
-rw-r--r-- 1 root root  35K Jul 12 17:51 COPYING
-rw-r--r-- 1 root root 6.5K Jul 12 17:51 PKG-INFO
-rw-r--r-- 1 root root 4.6K Jul 12 17:51 README.md
-rwxr--r-- 1 root root 3.0K Jul 12 17:51 build.sh
drwxr-xr-x 1 root root   92 Apr 17  2021 certs
drwxr-xr-x 1 root root  572 Jul 24 23:45 conf
drwxr-xr-x 1 root root   54 Jul 24 23:45 etc
-rw-r--r-- 1 root root 4.4K Jul 24 23:45 poetry-install.txt
-rw-r--r-- 1 root root  34K Jul 12 17:51 poetry.lock
-rw-r--r-- 1 root root   32 Jul 12 17:51 poetry.toml
-rw-r--r-- 1 root root 5.3K Jul 12 17:51 pyproject.toml
-rw-r--r-- 1 root root 520K Jul 12 17:51 rockstor-jslibs.tar.gz
-rw-r--r-- 1 root root   89 Jul 12 17:51 rockstor-jslibs.tar.gz.sha256sum
-rw-r--r-- 1 root root 320K Feb 13 23:31 rockstor-tasks-huey.db
drwxr-xr-x 1 root root   16 Jul 12 17:51 src
drwxr-xr-x 1 root root    6 Jul 12 17:51 var
NAS001:~ # cat /opt/rockstor/poetry-install.txt
The currently activated Python version 3.6.15 is not supported by the project (~                                                                                                                                                                                                                                                                                                                                                                                                             2.7).
Trying to find and use a compatible version.
Using python2 (2.7.18)
Creating virtualenv rockstor in /opt/rockstor/.venv
Installing dependencies from lock file

Package operations: 29 installs, 0 updates, 0 removals

  • Installing certifi (2021.10.8)
  • Installing chardet (4.0.0)
  • Installing greenlet (2.0.2)
  • Installing idna (2.10)
  • Installing pytz (2022.6)
  • Installing six (1.16.0)
  • Installing urllib3 (1.26.12)
  • Installing django (1.11.29)
  • Installing gevent (1.1.2)
  • Installing oauthlib (3.1.0)
  • Installing python-engineio (2.3.2)
  • Installing requests (2.27.1)
  • Installing dbus-python (1.2.18)
  • Installing distro (1.6.0)
  • Installing django-braces (1.13.0)
  • Installing django-oauth-toolkit (1.1.2)
  • Installing gevent-websocket (0.9.5)
  • Installing django-pipeline (1.6.9)
  • Installing gunicorn (19.10.0)
  • Installing djangorestframework (3.9.3)
  • Installing huey (2.3.0)
  • Installing mock (1.0.1)
  • Installing psutil (5.9.4)
  • Installing psycogreen (1.0)
  • Installing psycopg2 (2.8.6)
  • Installing python-socketio (1.6.0)
  • Installing pyzmq (19.0.2)
  • Installing supervisor (4.2.4)

  EnvCommandError

  Command ['/opt/rockstor/.venv/bin/pip', 'install', '--no-deps', 'file:///root/                                                                                                                                                                                                                                                                                                                                                                                                             .cache/pypoetry/artifacts/78/6c/1c/efa6250f008a1e079eda9b4ea03c74050bbcedd7264d3                                                                                                                                                                                                                                                                                                                                                                                                             72fd05febc42f/psycopg2-2.8.6.tar.gz'] errored with the following return code 1,                                                                                                                                                                                                                                                                                                                                                                                                              and output:
  DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Plea                                                                                                                                                                                                                                                                                                                                                                                                             se upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop                                                                                                                                                                                                                                                                                                                                                                                                              support for Python 2.7 in January 2021. More details about Python 2 support in                                                                                                                                                                                                                                                                                                                                                                                                              pip can be found at https://pip.pypa.io/en/latest/development/release-process/#p                                                                                                                                                                                                                                                                                                                                                                                                             ython-2-support pip 21.0 will remove support for this functionality.
  Processing /root/.cache/pypoetry/artifacts/78/6c/1c/efa6250f008a1e079eda9b4ea0                                                                                                                                                                                                                                                                                                                                                                                                             3c74050bbcedd7264d372fd05febc42f/psycopg2-2.8.6.tar.gz
      ERROR: Command errored out with exit status 1:
       command: /opt/rockstor/.venv/bin/python -c 'import sys, setuptools, token                                                                                                                                                                                                                                                                                                                                                                                                             ize; sys.argv[0] = '"'"'/tmp/pip-req-build-V2jj86/setup.py'"'"'; __file__='"'"'/                                                                                                                                                                                                                                                                                                                                                                                                             tmp/pip-req-build-V2jj86/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)                                                                                                                                                                                                                                                                                                                                                                                                             (__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(co                                                                                                                                                                                                                                                                                                                                                                                                             mpile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-inf                                                                                                                                                                                                                                                                                                                                                                                                             o-OKTQvi
           cwd: /tmp/pip-req-build-V2jj86/
      Complete output (23 lines):
      running egg_info
      creating /tmp/pip-pip-egg-info-OKTQvi/psycopg2.egg-info
      writing /tmp/pip-pip-egg-info-OKTQvi/psycopg2.egg-info/PKG-INFO
      writing top-level names to /tmp/pip-pip-egg-info-OKTQvi/psycopg2.egg-info/                                                                                                                                                                                                                                                                                                                                                                                                             top_level.txt
      writing dependency_links to /tmp/pip-pip-egg-info-OKTQvi/psycopg2.egg-info                                                                                                                                                                                                                                                                                                                                                                                                             /dependency_links.txt
      writing manifest file '/tmp/pip-pip-egg-info-OKTQvi/psycopg2.egg-info/SOUR                                                                                                                                                                                                                                                                                                                                                                                                             CES.txt'

      Error: pg_config executable not found.

      pg_config is required to build psycopg2 from source.  Please add the direc                                                                                                                                                                                                                                                                                                                                                                                                             tory
      containing pg_config to the $PATH or specify the full executable path with                                                                                                                                                                                                                                                                                                                                                                                                              the
      option:

          python setup.py build_ext --pg-config /path/to/pg_config build ...

      or with the pg_config option in 'setup.cfg'.

      If you prefer to avoid building psycopg2 from source, please install the P                                                                                                                                                                                                                                                                                                                                                                                                             yPI
      'psycopg2-binary' package instead.

      For further information please check the 'doc/src/install.rst' file (also                                                                                                                                                                                                                                                                                                                                                                                                              at
      <https://www.psycopg.org/docs/install.html>).

      ----------------------------------------
  ERROR: Command errored out with exit status 1: python setup.py egg_info Check                                                                                                                                                                                                                                                                                                                                                                                                              the logs for full command output.


  at ~/.local/share/pypoetry/venv/lib64/python3.6/site-packages/poetry/utils/env                                                                                                                                                                                                                                                                                                                                                                                                             .py:1195 in _run
      1191│                 output = subprocess.check_output(
      1192│                     cmd, stderr=subprocess.STDOUT, **kwargs
      1193│                 )
      1194│         except CalledProcessError as e:
    → 1195│             raise EnvCommandError(e, input=input_)
      1196│
      1197│         return decode(output)
      1198│
      1199│     def execute(self, bin, *args, **kwargs):

  • Installing urlobject (2.1.1)

so looks like a problem with the PostgresSQL database adapter for phyton, that could not be installed propperly.

3 Likes

same root cause as in Upgrade to 5.0.1-0 from 4.6.1 rockstor fails

NAS001:~ # which pg_config
which: no pg_config in (/sbin:/usr/sbin:/usr/local/sbin:/root/.local/bin:/root/bin:/usr/local/bin:/usr/bin:/bin:/usr/lib/mit/bin:/usr/lib/mit/sbin)
NAS001:~ # zypper info postgresql13-server-devel
Retrieving repository 'Netdata Edge' metadata .......................................................................................................................................................................................[done]
Building repository 'Netdata Edge' cache ............................................................................................................................................................................................[done]
Retrieving repository 'repo-backports-update' metadata ..............................................................................................................................................................................[done]
Building repository 'repo-backports-update' cache ...................................................................................................................................................................................[done]
Retrieving repository 'repo-sle-update' metadata ....................................................................................................................................................................................[done]
Building repository 'repo-sle-update' cache .........................................................................................................................................................................................[done]
Loading repository data...
Reading installed packages...


Information for package postgresql13-server-devel:
--------------------------------------------------
Repository     : repo-sle-update
Name           : postgresql13-server-devel
Version        : 13.11-150200.5.40.1
Arch           : x86_64
Vendor         : SUSE LLC <https://www.suse.com/>
Installed Size : 6.0 MiB
Installed      : Yes (automatically)
Status         : up-to-date
Source package : postgresql13-13.11-150200.5.40.1.src
Upstream URL   : https://www.postgresql.org/
Summary        : PostgreSQL server development header files and utilities
Description    :
    PostgreSQL is an advanced object-relational database management system
    that supports an extended subset of the SQL standard, including
    transactions, foreign keys, subqueries, triggers, and user-defined
    types and functions.

    This package contains the header files and libraries needed to compile
    C extensions that link into the PostgreSQL server. For building client
    applications, see the postgresql13-devel package.

Correct… As detailed in that thread, this seems like an issue related to some potential changes in how the pg_config binary was provided in the upstream packages, maybe related to systems that were upgraded from Leap 15.3 to 15.4 but I personally do not know for sure. I’ve laid out a few ideas on trying to figure that one out in that other thread.

2 Likes

Hello again @anborn,

As you may have seen in the other linked thread, the situation there has been resolved, so we can safely try the same for you.
See the following post: Upgrade to 5.0.1-0 from 4.6.1 rockstor fails - #16 by Flox
In particular, let’s see what versions of postgresql you have installed:

zypper search --installed-only | grep postgresql

If you are in the same situation as the other post, you should have postgresql13 and another one (maybe 14?).

In this case, let’s try to re-set the alternatives config for postgresql:

update-alternatives --config postgresql

If you do not see a line with postgresql13, then cancel by pressing Ctrl + C, otherwise, enter the number of the line corresponding to postgresql13.

Please report back with the outputs of each command if possible to make sure it all looks at expected.

2 Likes

Hi @Flox ,
I now also switched postgressql config to version 13 and system is back again :wink:
The only difference i saw is that I have the option to set 10, 13 and 14.
with setting it to postgresql13 it by manual mode, it works as expected after forced reinstalling of rockstor stable.


  Selection    Path                   Priority   Status
------------------------------------------------------------
  0            /usr/lib/postgresql14   140       auto mode
  1            /usr/lib/postgresql10   100       manual mode
* 2            /usr/lib/postgresql13   130       manual mode
  3            /usr/lib/postgresql14   140       manual mode

Since version 14 has the highest priority of 140 it will be chosen in auto mode imho.

Thank you for your help!

3 Likes

Glad to read that!

That was exactly what @phillxnet was thinking as well :slight_smile:

2 Likes