Build instructions

vesper1978 · April 5, 2019, 5:52pm

@jcdick1,

Have you tried what’s in:

http://rockstor.com/docs/contribute.html

It may not be fully up to date, but is probably a good place to start.

jcdick1 · April 5, 2019, 6:28pm

@phillxnet

Building on Tumbleweed, following all the instructions I could gather and assume apply.

Following the instructions provided in Contribute, the result from the command

/path/to/build_dir/bin/buildout -N -c /path/to/build_dir/buildout.cfg

is:

Setting socket time out to 120 seconds.
Develop: ‘/usr/opt/rockstor-core/.’
Getting distribution for ‘plone.recipe.command’.
Got plone.recipe.command 1.1.
Getting distribution for ‘djangorecipe==1.9’.
Got djangorecipe 1.9.
Getting distribution for ‘Django==1.8.16’.
warning: no previously-included files matching ‘pycache’ found under directory ‘’
warning: no previously-included files matching '.py[co]’ found under directory ‘’
Got Django 1.8.16.
Getting distribution for ‘zc.recipe.egg>=2.0.6’.
Got zc.recipe.egg 2.0.7.
Getting distribution for ‘collective.recipe.template’.
Got collective.recipe.template 2.1.
Getting distribution for ‘hexagonit.recipe.download’.
Got hexagonit.recipe.download 1.7.1.
Getting distribution for ‘collective.recipe.cmd’.
warning: no previously-included files matching '.pyc’ found anywhere in distribution
warning: no previously-included files found matching ‘bootstrap.py’
warning: no previously-included files found matching ‘buildout.cfg’
warning: no previously-included files found matching ‘tox.ini’
Got collective.recipe.cmd 0.11.
Unused options for buildout: ‘include-site-packages’.
Installing stop-rockstor.
stop-rockstor: Running ‘systemctl stop rockstor’
Failed to stop rockstor.service: Unit rockstor.service not loaded.
Unused options for stop-rockstor: ‘update-command’.
Installing django.
Getting distribution for ‘distro’.
/usr/lib64/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: ‘long_description_content_type’
warnings.warn(msg)
warning: no files found matching ‘CHANGES’
warning: no previously-included files matching ‘.py[co]’ found anywhere in distribution
zip_safe flag not set; analyzing archive contents…
Got distro 1.4.0.
Getting distribution for ‘six==1.10.0’.
no previously-included directories found matching ‘documentation/_build’
zip_safe flag not set; analyzing archive contents…
six: module references path
Got six 1.10.0.
Getting distribution for ‘requests==1.1.0’.
Got requests 1.1.0.
Getting distribution for ‘pyzmq==15.0.0’.
warning: no files found matching ‘tox.ini’
no previously-included directories found matching ‘docs/build’
no previously-included directories found matching ‘docs/gh-pages’
warning: no previously-included files found matching 'bundled/zeromq/src/Makefile’
warning: no previously-included files found matching ‘bundled/zeromq/src/platform.hpp’
warning: no previously-included files found matching ‘zmq/libzmq*’
warning: no previously-included files matching ‘pycache/’ found anywhere in distribution
warning: no previously-included files matching '.deps/’ found anywhere in distribution
warning: no previously-included files matching ‘.so’ found anywhere in distribution
warning: no previously-included files matching '.pyd’ found anywhere in distribution
warning: no previously-included files matching ‘.git*’ found anywhere in distribution
warning: no previously-included files matching ‘.DS_Store’ found anywhere in distribution
warning: no previously-included files matching ‘.mailmap’ found anywhere in distribution
warning: no previously-included files matching ‘Makefile.am’ found anywhere in distribution
warning: no previously-included files matching ‘Makefile.in’ found anywhere in distribution
Did not find libzmq via pkg-config.
{‘libraries’: [‘zmq’], ‘extra_link_args’: [], ‘runtime_library_dirs’: [], ‘library_dirs’: [], ‘include_dirs’: []}

Configure: Autodetecting ZMQ settings…
Custom ZMQ dir:
/tmp/easy_install-lxpF9i/pyzmq-15.0.0/temp/timer_createWqOE0I.c:1:1: warning: return type defaults to ‘int’ [-Wimplicit -int]
main (int argc, char **argv) {
^~~~
/tmp/easy_install-lxpF9i/pyzmq-15.0.0/temp/timer_createWqOE0I.c: In function ‘main’:
/tmp/easy_install-lxpF9i/pyzmq-15.0.0/temp/timer_createWqOE0I.c:2:5: warning: implicit declaration of function ‘timer_c reate’ [-Wimplicit-function-declaration]
timer_create();
^~~~~~~~~~~~
/usr/lib64/gcc/x86_64-suse-linux/8/…/…/…/…/x86_64-suse-linux/bin/ld: build/temp.linux-x86_64-2.7/scratch/tmp/easy_i nstall-lxpF9i/pyzmq-15.0.0/temp/timer_createWqOE0I.o: in function main': timer_createWqOE0I.c:(.text+0x15): undefined reference totimer_create’
collect2: error: ld returned 1 exit status
build/temp.linux-x86_64-2.7/scratch/vers.c:4:10: fatal error: zmq.h: No such file or directory
#include “zmq.h”
^~~~~~~
compilation terminated.

error: command ‘gcc’ failed with exit status 1

Failed with default libzmq, trying again with /usr/local
{‘libraries’: [‘zmq’], ‘extra_link_args’: [], ‘runtime_library_dirs’: [], ‘library_dirs’: [’/usr/local/lib’], ‘include_ dirs’: [’/usr/local/include’]}

Configure: Autodetecting ZMQ settings…
Custom ZMQ dir: /usr/local
/tmp/easy_install-lxpF9i/pyzmq-15.0.0/temp/timer_createm0fPPq.c:1:1: warning: return type defaults to ‘int’ [-Wimplicit -int]
main (int argc, char **argv) {
^~~~
/tmp/easy_install-lxpF9i/pyzmq-15.0.0/temp/timer_createm0fPPq.c: In function ‘main’:
/tmp/easy_install-lxpF9i/pyzmq-15.0.0/temp/timer_createm0fPPq.c:2:5: warning: implicit declaration of function ‘timer_c reate’ [-Wimplicit-function-declaration]
timer_create();
^~~~~~~~~~~~
Assembler messages:
Fatal error: can’t create build/temp.linux-x86_64-2.7/scratch/tmp/easy_install-lxpF9i/pyzmq-15.0.0/temp/timer_createm0f PPq.o: No such file or directory
build/temp.linux-x86_64-2.7/scratch/vers.c:4:10: fatal error: zmq.h: No such file or directory
#include “zmq.h”
^~~~~~~
compilation terminated.

error: command ‘gcc’ failed with exit status 1

Warning: Couldn’t find an acceptable libzmq on the system.

If you expected pyzmq to link against an installed libzmq, please check to make sure:

You have a C compiler installed
* A development version of Python is installed (including headers)
* A development version of ZMQ >= 3.2 is installed (including headers)
* If ZMQ is not in a default location, supply the argument --zmq=
* If you did recently install ZMQ to a default location,
try rebuilding the ld cache with sudo ldconfig
or specify zmq’s location with --zmq=/usr/local

You can skip all this detection/waiting nonsense if you know
you want pyzmq to bundle libzmq as an extension by passing:

--zmq=bundled

I will now try to build libzmq as a Python extension
unless you interrupt me (^C) in the next 10 seconds…

1…

Using bundled libzmq
already have bundled/zeromq
attempting ./configure to generate platform.hpp
Warning: failed to configure libzmq:
/bin/sh: ./configure: No such file or directory

staging platform.hpp from: /tmp/easy_install-lxpF9i/pyzmq-15.0.0/buildutils/include_linux

checking for timer_create
/tmp/easy_install-lxpF9i/pyzmq-15.0.0/temp/timer_createPiq8AW.c:1:1: warning: return type defaults to ‘int’ [-Wimplicit -int]
main (int argc, char **argv) {
^~~~
/tmp/easy_install-lxpF9i/pyzmq-15.0.0/temp/timer_createPiq8AW.c: In function ‘main’:
/tmp/easy_install-lxpF9i/pyzmq-15.0.0/temp/timer_createPiq8AW.c:2:5: warning: implicit declaration of function ‘timer_c reate’ [-Wimplicit-function-declaration]
timer_create();
^~~~~~~~~~~~
/usr/lib64/gcc/x86_64-suse-linux/8/…/…/…/…/x86_64-suse-linux/bin/ld: build/temp.linux-x86_64-2.7/tmp/easy_install-l xpF9i/pyzmq-15.0.0/temp/timer_createPiq8AW.o: in function main': timer_createPiq8AW.c:(.text+0x15): undefined reference totimer_create’
collect2: error: ld returned 1 exit status
no timer_create, linking librt

Using bundled libsodium
already have bundled/libsodium
already have crypto_scalarmult_curve25519.h
already have crypto_stream_salsa20.h
staging /tmp/easy_install-lxpF9i/pyzmq-15.0.0/buildutils/include_sodium/version.h to bundled/libsodium/src/libsodium/in clude/sodium/version.h

bundled/libsodium/src/libsodium/randombytes/salsa20/randombytes_salsa20_random.c: In function ‘randombytes_salsa20_rand om_getword’:
bundled/libsodium/src/libsodium/randombytes/salsa20/randombytes_salsa20_random.c:311:14: warning: variable ‘ret’ set bu t not used [-Wunused-but-set-variable]
int ret;
^~~
bundled/libsodium/src/libsodium/randombytes/salsa20/randombytes_salsa20_random.c: In function ‘randombytes_salsa20_rand om_buf’:
bundled/libsodium/src/libsodium/randombytes/salsa20/randombytes_salsa20_random.c:371:12: warning: variable ‘ret’ set bu t not used [-Wunused-but-set-variable]
int ret;
^~~
bundled/libsodium/src/libsodium/sodium/utils.c:45:15: warning: ‘page_size’ defined but not used [-Wunused-variable]
static size_t page_size;
^~~~~~~~~
zmq/backend/cython/socket.c: In function ‘__pyx_f_3zmq_7backend_6cython_6socket__send_copy.constprop’:
zmq/backend/cython/socket.c:2587:15: warning: ‘__pyx_v_rc2’ may be used uninitialized in this function [-Wmaybe-uninitialized]
__pyx_t_2 = __pyx_f_3zmq_7backend_6cython_7checkrc__check_rc(__pyx_v_rc2); if (unlikely(__pyx_t_2 == -1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 204; __pyx_clineno = LINE; goto __pyx_L1_error;}
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
File “build/bdist.linux-x86_64/egg/zmq/tests/_test_asyncio.py”, line 37
yield from a.send(b’hi’)
^
SyntaxError: invalid syntax

File “/usr/opt/rockstor-core/eggs/tmpbe05Tf/pyzmq-15.0.0-py2.7-linux-x86_64.egg/zmq/tests/_test_asyncio.py”, line 37
yield from a.send(b’hi’)
^
SyntaxError: invalid syntax

Got pyzmq 15.0.0.
Getting distribution for ‘pytz==2014.3’.
Got pytz 2014.3.
Getting distribution for ‘python-socketio==1.6.0’.
warning: no files found matching ‘README.md’
Got python-socketio 1.6.0.
Getting distribution for ‘psycopg2==2.7.4’.

Error: pg_config executable not found.

pg_config is required to build psycopg2 from source. Please add the directory
containing pg_config to the $PATH or specify the full executable path with the
option:

python setup.py build_ext --pg-config /path/to/pg_config build …

or with the pg_config option in ‘setup.cfg’.

If you prefer to avoid building psycopg2 from source, please install the PyPI
‘psycopg2-binary’ package instead.

For further information please check the ‘doc/src/install.rst’ file (also at
http://initd.org/psycopg/docs/install.html).

error: Setup script exited with 1
An error occurred when trying to install /tmp/dist/psycopg2-2.7.4.tar.gz. Look above this message for any errors that were output by easy_install.
While:
Installing django.
Getting distribution for ‘psycopg2==2.7.4’.

An internal error occurred due to a bug in either zc.buildout or in a
recipe being used:
Traceback (most recent call last):
File “/usr/opt/rockstor-core/eggs/zc.buildout-2.13.1-py2.7.egg/zc/buildout/buildout.py”, line 2174, in main
getattr(buildout, command)(args)
File “/usr/opt/rockstor-core/eggs/zc.buildout-2.13.1-py2.7.egg/zc/buildout/buildout.py”, line 817, in install
installed_files = self[part]._call(recipe.install)
File “/usr/opt/rockstor-core/eggs/zc.buildout-2.13.1-py2.7.egg/zc/buildout/buildout.py”, line 1603, in _call
return f()
File “/usr/opt/rockstor-core/eggs/djangorecipe-1.9-py2.7.egg/djangorecipe/recipe.py”, line 67, in install
requirements, ws = self.egg.working_set([‘djangorecipe’])
File “/usr/opt/rockstor-core/eggs/zc.recipe.egg-2.0.7-py2.7.egg/zc/recipe/egg/egg.py”, line 87, in working_set
allow_unknown_extras=bool_option(buildout_section, ‘allow-unknown-extras’)
File “/usr/opt/rockstor-core/eggs/zc.recipe.egg-2.0.7-py2.7.egg/zc/recipe/egg/egg.py”, line 168, in _working_set
allow_unknown_extras=allow_unknown_extras)
File “/usr/opt/rockstor-core/eggs/zc.buildout-2.13.1-py2.7.egg/zc/buildout/easy_install.py”, line 957, in install
return installer.install(specs, working_set)
File “/usr/opt/rockstor-core/eggs/zc.buildout-2.13.1-py2.7.egg/zc/buildout/easy_install.py”, line 730, in install
for dist in self._get_dist(req, ws):
File “/usr/opt/rockstor-core/eggs/zc.buildout-2.13.1-py2.7.egg/zc/buildout/easy_install.py”, line 574, in _get_dist
dists = [_move_to_eggs_dir_and_compile(dist, self._dest)]
File “/usr/opt/rockstor-core/eggs/zc.buildout-2.13.1-py2.7.egg/zc/buildout/easy_install.py”, line 1740, in _move_to_eggs_dir_and_compile
[tmp_loc] = glob.glob(os.path.join(tmp_dest, ‘*’))
ValueError: need more than 0 values to unpack

So something is definitely missing from the instructions.

phillxnet · April 5, 2019, 8:21pm

@jcdick1 Hello again. Linking to your prior thread where I gave instructions pertinent to you query in this thread:

The build instructions are intended for a developer audience and as such are not really end user material. But yes we could do with including the normally indicated pre-requisite apps for building. But these are already indicated in for example the more specific doc for building on openSUSE which I referenced in your last thread. I.e. see my 2 references from that thread:

ie look to:

and the suggested pre-requisite packages as given in the latter:

Built on openSUSE dev notes and status

Packages for Building Rockstor from source

zypper --non-interactive install gcc python-devel gcc-c++ postgresql-devel libblkid-devel python-distro firewalld

I expect you only installed those packages specified in “Packages for Rockstor function” but you are aiming at the building from source instructions.

Let us know how this goes and please submit a pull request to fill out our dev docs upon finding something missing. But also note that these instructions are specifically for developers and not really end user friendly. I can’t spend much time helping you through them as your are not their intended audience. But at the same time this is a definite improvement and is long outstanding. Hence me adding them in the openSUSE instructions, our intended future base (but not ready just yet).

If you wish to get further help on the build please paste the exact steps you take along with a cut and paste the the exact commands you run to get the the problem state you report. Development is exact and the rpm offering is for ease of install / end user. What we need to improve the developer docs is all steps that were taken so that the error you received can be reproduced. So if posting again on this issue please specify how and what you installed so it can be reproduced exactly.

But yes we definitely need to make things as easy as possible to help with developer on-boarding which is hard enough at the best of times.

You are rather stuck between a rock and a hard place (from your linked prior threads context) and a fresh set of eyes is always useful on our docs so thanks for the pointer, although I did provide the required packages in my wiki entry we should also have them in the general dev docs.

I’ve created the following issue to address the docs issue:

https://github.com/rockstor/rockstor-doc/issues/201

Good luck with the build but do keep in mind that all developer instructions are intended to development purposes. We release via rpm to help end users, and the openSUSE function is far from on par currently with CentOS. But as I mentioned in your last linked thread it may be your only option. However you mention there a hardware issue found that was then deleted as a post. I’ll pick that one up in that thread. Also note that @vesper1978 contribution, although much appreciated, was in part an artifact of you not linking to your prior post as context, i.e. if you had linked they could have perhaps helped you further along having already seen our prior discussion.

Always tricky to know when to best split off a discussion though so lets just muddle through as best we can.

Hope that helps and that you the partial function of our build on openSUSE is enough to get you going until we start releasing rpms for that platform.

jcdick1 · April 5, 2019, 8:42pm

I just double-checked by doing a copy/paste of the command for both “Packages for Rockstor function” and “Packages for Building Rockstor from source,” with a “Nothing to do” result from both. So all of the stated required packages are installed.

I’ve created a /usr/opt directory to use in lieu of the RPM default /opt/rockstor.

In /usr/opt I ran

git clone https://github.com/rockstor/rockstor-core.git

When that was complete, I referred to the Dev section of the Contributing, using the “Build VM initial setup” as reference:

python ./rockstor-core/bootstrap.py -c ./rockstor-core/buildout.cfg

The result:

ez_setup.py is deprecated and when using it setuptools will be pinned to 33.1.1 since it’s the last version that supports setuptools self upgrade/installation, check https://github.com/pypa/setuptools/issues/581 for more info; use pip to install setuptools
Downloading https://pypi.io/packages/source/s/setuptools/setuptools-33.1.1.zip
Extracting in /tmp/tmpYSgSlM
Now working in /tmp/tmpYSgSlM/setuptools-33.1.1
Building a Setuptools egg in /tmp/tmpuLHSCs
warning: no files found matching ‘*’ under directory ‘setuptools/_vendor’
/tmp/tmpuLHSCs/setuptools-33.1.1-py2.7.egg
/tmp/tmpuLHSCs/setuptools-33.1.1-py2.7.egg/pkg_resources/init.py:184: RuntimeWarning: You have iterated over the result of pkg_resources.parse_version. This is a legacy behavior which is inconsistent with the new version class introduced in setuptools 8.0. In most cases, conversion to a tuple is unnecessary. For comparison of versions, sort the Version instances directly. If you have another use case requiring the tuple, please file a bug with the setuptools project describing that need.
no previously-included directories found matching ‘doc’
no previously-included directories found matching ‘old-tutorial’
Setting socket time out to 120 seconds.
Creating directory ‘/usr/opt/rockstor-core/eggs’.
Creating directory ‘/usr/opt/rockstor-core/bin’.
Creating directory ‘/usr/opt/rockstor-core/parts’.
Creating directory ‘/usr/opt/rockstor-core/develop-eggs’.
Generated script ‘/usr/opt/rockstor-core/bin/buildout’.

Based on the error statement regarding ez_setup.py, I don’t know that installing the openSUSE python-setuptools (v40.8.0) package would resolve it.

Moving on then to

./rockstor-core/bin/buildout -N -c ./rockstor-core/buildout.cfg

Results in the previous mentioned errors.

phillxnet · April 5, 2019, 9:05pm

@jcdick1 Thanks for the double check on those package requirements.

On creating the openSUSE dev doc I used cut paste with a clean install so we may have had an upstream change in the mean time. Can’t have a look directly but will have a look again soon.

I’ve yet to establish an automated build in my jenkins setup for openSUSE so can’t check on any changes over time from there. But all in good time.

@Flox or @luke may be in a better time zone to help and have both successfully build on openSUSE although I only know of @Flox building on both Leap15.1 and Tumbleweed.

Thanks for persevering and I will try and take a look soon myself but do report back if you sort this. But the openSUSE build is not my current main focus as we are in pressing need to give stable subscribers the ability to edit their own appliance ids such that we inconvenience them less during re-install which is like also to be an element of our openSUSE move.

Oh well, many moving parts and only so much time in the day. I’ll re-test my wiki as soon as I can and update it accordingly and post back here there after.

jcdick1 · April 5, 2019, 9:40pm

I’m taking notes, so if I do manage to get it sorted, I’ll submit some kind of updated documentation.

phillxnet · April 7, 2019, 2:53pm

@jcdick1 Hello again.

I’ve just managed to reproduce your build issue on a recent Tumbleweed install that had previously build rockstor OK.

Getting distribution for 'psycopg2==2.7.4'.

Error: pg_config executable not found.

pg_config is required to build psycopg2 from source.

And it looks like upstream package changes so far as if we look to prior snapshots of root:

./.snapshots/35/snapshot/etc/alternatives/pg_config
./.snapshots/35/snapshot/usr/lib/postgresql10/bin/pg_config
./.snapshots/35/snapshot/usr/bin/pg_config

and the same through to snapshot 39 inclusive:

./.snapshots/39/snapshot/etc/alternatives/pg_config
./.snapshots/39/snapshot/usr/lib/postgresql10/bin/pg_config
./.snapshots/39/snapshot/usr/bin/pg_config

But my root snapshots numbered 40 and 41, and then the current root show only the following file:

/usr/lib/postgresql10/bin/pg_config
I.e.:

./.snapshots/40/snapshot/usr/lib/postgresql10/bin/pg_config
./.snapshots/41/snapshot/usr/lib/postgresql10/bin/pg_config
./usr/lib/postgresql10/bin/pg_config

Where the last working in path version of this file I have is:

ls -la ./.snapshots/39/snapshot/usr/bin/pg_config
./.snapshots/39/snapshot/usr/bin/pg_config -> /etc/alternatives/pg_config

a link to the alternatives system which in turn referenced our target:

ls -la ./.snapshots/39/snapshot/etc/alternatives/pg_config                                                                                         
./.snapshots/39/snapshot/etc/alternatives/pg_config -> /usr/lib/postgresql10/bin/pg_config

and from:

sudo zypper search --installed-only | grep postgres     
  

i  | postgresql                           | Basic Clients and Utilities for PostgreSQL                                         | package
i+ | postgresql-devel                     | PostgreSQL development header files and libraries                                  | package
i+ | postgresql-server                    | The Programs Needed to Create and Run a PostgreSQL Server                          | package
i  | postgresql10                         | Basic Clients and Utilities for PostgreSQL                                         | package
i  | postgresql10-devel                   | PostgreSQL development header files and libraries                                  | package
i  | postgresql10-server                  | The Programs Needed to Create and Run a PostgreSQL Server                          | package
i  | postgresql11                         | Basic Clients and Utilities for PostgreSQL                                         | package
i  | postgresql11-server                  | The Programs Needed to Create and Run a PostgreSQL Server                          | package

It would appear that Tumbleweed has now added a postgresql 11.

and from the alternatives system we have:

alternatives --display pip                                                                                                           
/usr/lib/postgresql10 
/usr/lib/postgresql11

and for comparison for an similar alternatives managed program ‘pip’ we have:

alternatives --display pip
pip - auto mode
  link best version is /usr/bin/pip3.7
  link currently points to /usr/bin/pip3.7
  link pip is /usr/bin/pip
/usr/bin/pip2.7 - priority 27
/usr/bin/pip3.7 - priority 37

and postgresql looks to have the same auto setting yet we don’t have our expected links:

alternatives --config postgresql
There are 2 choices for the alternative postgresql (providing /usr/lib/postgresql).

 Selection    Path                   Priority   Status
------------------------------------------------------------
* 0            /usr/lib/postgresql11   110       auto mode
 1            /usr/lib/postgresql10   100       manual mode
 2            /usr/lib/postgresql11   110       manual mode

Press <enter> to keep the current choice[*], or type selection number: 1
update-alternatives: using /usr/lib/postgresql10 to provide /usr/lib/postgresql (postgresql) in manual mode

Here I selected 1 as it’s what we have been using all along, at least in our openSUSE source builds.

And there after we now at least have an ‘in path’ reference to pg_config (note the last line):

find . -name pg_config
...
./.snapshots/39/snapshot/etc/alternatives/pg_config
./.snapshots/39/snapshot/usr/lib/postgresql10/bin/pg_config
./.snapshots/39/snapshot/usr/bin/pg_config
./.snapshots/40/snapshot/usr/lib/postgresql10/bin/pg_config
./.snapshots/41/snapshot/usr/lib/postgresql10/bin/pg_config
./usr/lib/postgresql10/bin/pg_config
./usr/bin/pg_config
...

and then we have a successful build:

...
Installing start-rockstor.
start-rockstor: Running 'systemctl stop rockstor-pre &&
systemctl start rockstor'
Unused options for start-rockstor: 'update-command'.
tumbleweed:/opt #

and a shiny new badge:
recent-tumbleweed

Hope that helps and do remember to look at the outstanding GitHub issues we have open against our openSUSE target.

jcdick1 · April 9, 2019, 11:16pm

I managed to get where you were, in relation to the pg_config, but I get a ton of syntax errors, and ultimately, a:

Installing start-rockstor.
start-rockstor: Running ‘systemctl stop rockstor-pre &&
systemctl start rockstor’
A dependency job for rockstor.service failed. See ‘journalctl -xe’ for details.
Unused options for start-rockstor: ‘update-command’.

I did go back and install zeromq-devel and libzmq5 packages to see if that made things any easier, since it calls on libzmq but it wasn’t in the required packages list, and tries to build it itself.

Here it is, in its entirety:

user@host>./rockstor-core/bin/buildout -N -c ./rockstor-core/buildout.cfg
Setting socket time out to 120 seconds.
Develop: ‘/usr/opt/rockstor-core/.’
Getting distribution for ‘plone.recipe.command’.
Got plone.recipe.command 1.1.
Getting distribution for ‘djangorecipe==1.9’.
Got djangorecipe 1.9.
Getting distribution for ‘Django==1.8.16’.
warning: no previously-included files matching ‘pycache’ found under directory ‘’
warning: no previously-included files matching '.py[co]’ found under directory ‘’
Got Django 1.8.16.
Getting distribution for ‘zc.recipe.egg>=2.0.6’.
Got zc.recipe.egg 2.0.7.
Getting distribution for ‘collective.recipe.template’.
Got collective.recipe.template 2.1.
Getting distribution for ‘hexagonit.recipe.download’.
Got hexagonit.recipe.download 1.7.1.
Getting distribution for ‘collective.recipe.cmd’.
warning: no previously-included files matching '.pyc’ found anywhere in distribution
warning: no previously-included files found matching ‘bootstrap.py’
warning: no previously-included files found matching ‘buildout.cfg’
warning: no previously-included files found matching ‘tox.ini’
Got collective.recipe.cmd 0.11.
Unused options for buildout: ‘include-site-packages’.
Installing stop-rockstor.
stop-rockstor: Running ‘systemctl stop rockstor’
Unused options for stop-rockstor: ‘update-command’.
Installing django.
Getting distribution for ‘six==1.10.0’.
no previously-included directories found matching ‘documentation/_build’
zip_safe flag not set; analyzing archive contents…
six: module references path
Got six 1.10.0.
Getting distribution for ‘requests==1.1.0’.
Got requests 1.1.0.
Getting distribution for ‘pyzmq==15.0.0’.
warning: no files found matching ‘tox.ini’
no previously-included directories found matching ‘docs/build’
no previously-included directories found matching ‘docs/gh-pages’
warning: no previously-included files found matching ‘bundled/zeromq/src/Makefile*’
warning: no previously-included files found matching ‘bundled/zeromq/src/platform.hpp’
warning: no previously-included files found matching ‘zmq/libzmq*’
warning: no previously-included files matching ‘pycache/’ found anywhere in distribution
warning: no previously-included files matching '.deps/’ found anywhere in distribution
warning: no previously-included files matching ‘.so’ found anywhere in distribution
warning: no previously-included files matching '.pyd’ found anywhere in distribution
warning: no previously-included files matching ‘.git*’ found anywhere in distribution
warning: no previously-included files matching ‘.DS_Store’ found anywhere in distribution
warning: no previously-included files matching ‘.mailmap’ found anywhere in distribution
warning: no previously-included files matching ‘Makefile.am’ found anywhere in distribution
warning: no previously-included files matching ‘Makefile.in’ found anywhere in distribution
Settings obtained from pkg-config: {‘libraries’: [u’zmq’], ‘library_dirs’: [], ‘include_dirs’: [u’/usr/include/pgm-5.2’]}
{‘libraries’: [u’zmq’], ‘extra_link_args’: [], ‘runtime_library_dirs’: [], ‘library_dirs’: [], ‘include_dirs’: [u’/usr/include/pgm-5.2’]}

Configure: Autodetecting ZMQ settings…
Custom ZMQ dir:
/tmp/easy_install-NQS48b/pyzmq-15.0.0/temp/timer_createPAA9L2.c:1:1: warning: return type defaults to âintâ [-Wimplicit-int]
main (int argc, char **argv) {
^~~~
/tmp/easy_install-NQS48b/pyzmq-15.0.0/temp/timer_createPAA9L2.c: In function âmainâ:
/tmp/easy_install-NQS48b/pyzmq-15.0.0/temp/timer_createPAA9L2.c:2:5: warning: implicit declaration of function âtimer_createâ [-Wimplicit-function-declaration]
timer_create();
^~~~~~~~~~~~
/usr/lib64/gcc/x86_64-suse-linux/8/…/…/…/…/x86_64-suse-linux/bin/ld: build/temp.linux-x86_64-2.7/scratch/tmp/easy_install-NQS48b/pyzmq-15.0.0/temp/timer_createPAA9L2.o: in function main': timer_createPAA9L2.c:(.text+0x15): undefined reference totimer_create’
collect2: error: ld returned 1 exit status
ZMQ version detected: 4.3.1
Warning: Detected ZMQ version: 4.3.1. Some new features in libzmq may not be exposed by pyzmq.

In file included from zmq/devices/monitoredqueue.c:241:
/usr/include/zmq_utils.h:45:9: note: #pragma message: Warning: zmq_utils.h is deprecated. All its functionality is provided by zmq.h.
#pragma message(
^~~~~~~
In file included from zmq/backend/cython/_device.c:241:
/usr/include/zmq_utils.h:45:9: note: #pragma message: Warning: zmq_utils.h is deprecated. All its functionality is provided by zmq.h.
#pragma message(
^~~~~~~
In file included from zmq/backend/cython/_poll.c:243:
/usr/include/zmq_utils.h:45:9: note: #pragma message: Warning: zmq_utils.h is deprecated. All its functionality is provided by zmq.h.
#pragma message(
^~~~~~~
In file included from zmq/backend/cython/_version.c:241:
/usr/include/zmq_utils.h:45:9: note: #pragma message: Warning: zmq_utils.h is deprecated. All its functionality is provided by zmq.h.
#pragma message(
^~~~~~~
In file included from zmq/backend/cython/constants.c:241:
/usr/include/zmq_utils.h:45:9: note: #pragma message: Warning: zmq_utils.h is deprecated. All its functionality is provided by zmq.h.
#pragma message(
^~~~~~~
In file included from zmq/backend/cython/context.c:243:
/usr/include/zmq_utils.h:45:9: note: #pragma message: Warning: zmq_utils.h is deprecated. All its functionality is provided by zmq.h.
#pragma message(
^~~~~~~
In file included from zmq/backend/cython/error.c:241:
/usr/include/zmq_utils.h:45:9: note: #pragma message: Warning: zmq_utils.h is deprecated. All its functionality is provided by zmq.h.
#pragma message(
^~~~~~~
In file included from zmq/backend/cython/message.c:244:
/usr/include/zmq_utils.h:45:9: note: #pragma message: Warning: zmq_utils.h is deprecated. All its functionality is provided by zmq.h.
#pragma message(
^~~~~~~
In file included from zmq/backend/cython/socket.c:246:
/usr/include/zmq_utils.h:45:9: note: #pragma message: Warning: zmq_utils.h is deprecated. All its functionality is provided by zmq.h.
#pragma message(
^~~~~~~
zmq/backend/cython/socket.c: In function â__pyx_f_3zmq_7backend_6cython_6socket__send_copy.constpropâ:
zmq/backend/cython/socket.c:2587:15: warning: â__pyx_v_rc2â may be used uninitialized in this function [-Wmaybe-uninitialized]
__pyx_t_2 = __pyx_f_3zmq_7backend_6cython_7checkrc__check_rc(__pyx_v_rc2); if (unlikely(__pyx_t_2 == -1)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 204; __pyx_clineno = LINE; goto __pyx_L1_error;}
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from zmq/backend/cython/utils.c:241:
/usr/include/zmq_utils.h:45:9: note: #pragma message: Warning: zmq_utils.h is deprecated. All its functionality is provided by zmq.h.
#pragma message(
^~~~~~~
File “build/bdist.linux-x86_64/egg/zmq/tests/_test_asyncio.py”, line 37
yield from a.send(b’hi’)
^
SyntaxError: invalid syntax

File “/usr/opt/rockstor-core/eggs/tmpuRhVKP/pyzmq-15.0.0-py2.7-linux-x86_64.egg/zmq/tests/_test_asyncio.py”, line 37
yield from a.send(b’hi’)
^
SyntaxError: invalid syntax

Got pyzmq 15.0.0.
Getting distribution for ‘pytz==2014.3’.
Got pytz 2014.3.
Getting distribution for ‘python-socketio==1.6.0’.
warning: no files found matching ‘README.md’
Got python-socketio 1.6.0.
Getting distribution for ‘psycopg2==2.7.4’.
zip_safe flag not set; analyzing archive contents…
psycopg2.tests.test_module: module references file
psycopg2.tests.test_types_basic: module references file
Got psycopg2 2.7.4.
Getting distribution for ‘psycogreen==1.0’.
zip_safe flag not set; analyzing archive contents…
Got psycogreen 1.0.
Getting distribution for ‘psutil==3.3.0’.
warning: no previously-included files matching ‘*’ found under directory ‘docs/_build’
warning: manifest_maker: MANIFEST.in, line 18: ‘recursive-include’ expects …

zip_safe flag not set; analyzing archive contents…
psutil.init: module references file
Got psutil 3.3.0.
Getting distribution for ‘mock==1.0.1’.
warning: no files found matching ‘.png’ under directory ‘docs’
warning: no files found matching '.css’ under directory ‘docs’
warning: no files found matching ‘.html’ under directory ‘docs’
warning: no files found matching '.js’ under directory ‘docs’
zip_safe flag not set; analyzing archive contents…
Got mock 1.0.1.
Getting distribution for ‘gevent-websocket==0.9.5’.
Got gevent-websocket 0.9.5.
Getting distribution for ‘gevent==1.1.2’.
no previously-included directories found matching ‘/pycache’
warning: no previously-included files matching '.so’ found anywhere in distribution
warning: no previously-included files matching ‘.o’ found anywhere in distribution
warning: no previously-included files matching ‘config.log’ found anywhere in distribution
warning: no previously-included files matching ‘config.status’ found anywhere in distribution
no previously-included directories found matching ‘doc/_build’
warning: no previously-included files matching '.pyc’ found anywhere in distribution
warning: no previously-included files matching ‘.coverage’ found under directory ‘greentest’
no previously-included directories found matching ‘greentest/htmlcov’
warning: no previously-included files matching ‘stamp-h?’ found under directory ‘c-ares’
warning: no previously-included files matching ‘ares_build.h.orig’ found under directory ‘c-ares’
no previously-included directories found matching ‘libev/.deps’
warning: no previously-included files matching ‘Makefile’ found under directory ‘libev’
warning: no previously-included files matching ‘libtool’ found under directory ‘libev’
warning: no previously-included files matching ‘stamp-h?’ found under directory ‘libev’
warning: no previously-included files matching ‘config.h’ found under directory ‘libev’
warning: no previously-included files matching ‘_corecffi.c’ found under directory ‘gevent’
warning: no previously-included files found matching ‘Makefile’
Running ‘(cd “/tmp/easy_install-mB7JTK/gevent-1.1.2/libev” && /bin/sh ./configure && cp config.h “$OLDPWD” ) > configure-output.txt’ in /tmp/easy_install-mB7JTK/gevent-1.1.2/build/temp.linux-x86_64-2.7/libev
In file included from gevent/libev.h:2,
from gevent/gevent.corecext.c:273:
libev/ev.c:483:48: warning: "/" within comment [-Wcomment]
/#define MIN_INTERVAL 0.00000095367431640625 /* 1/2**20, good till 2200 */

In file included from gevent/libev.h:2,
from gevent/gevent.corecext.c:273:
libev/ev.c:1692:31: warning: âev_default_loop_ptrâ initialized and declared âexternâ
EV_API_DECL struct ev_loop ev_default_loop_ptr = 0; / needs to be initialised to make it a definition despite extern */
^~~~~~~~~~~~~~~~~~~
In file included from gevent/libev.h:2,
from gevent/gevent.corecext.c:273:
libev/ev.c: In function âev_io_startâ:
libev/ev.c:3715:34: warning: suggest parentheses around arithmetic in operand of â|â [-Wparentheses]
fd_change (EV_A_ fd, w->events & EV__IOFDSET | EV_ANFD_REIFY);
^~~
libev/ev.c: At top level:
libev/ev.c:4956:27: warning: "/" within comment [-Wcomment]
/ EV_STAT 0x00001000 /* stat data changed */

libev/ev.c:4957:27: warning: "/" within comment [-Wcomment]
/ EV_EMBED 0x00010000 /* embedded event loop needs sweep */

In file included from gevent/libev.h:2,
from gevent/gevent.corecext.c:273:
libev/ev.c: In function âevpipe_writeâ:
libev/ev.c:2321:11: warning: ignoring return value of âwriteâ, declared with attribute warn_unused_result [-Wunused-result]
write (evpipe [1], &counter, sizeof (uint64_t));
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
libev/ev.c:2333:11: warning: ignoring return value of âwriteâ, declared with attribute warn_unused_result [-Wunused-result]
write (evpipe [1], &(evpipe [1]), 1);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
libev/ev.c: In function âpipecbâ:
libev/ev.c:2354:11: warning: ignoring return value of âreadâ, declared with attribute warn_unused_result [-Wunused-result]
read (evpipe [1], &counter, sizeof (uint64_t));
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
libev/ev.c:2368:11: warning: ignoring return value of âreadâ, declared with attribute warn_unused_result [-Wunused-result]
read (evpipe [0], &dummy, sizeof (dummy));
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Linking /tmp/easy_install-mB7JTK/gevent-1.1.2/build/lib.linux-x86_64-2.7/gevent/corecext.so to /tmp/easy_install-mB7JTK/gevent-1.1.2/gevent/corecext.so
Running ‘(cd “/tmp/easy_install-mB7JTK/gevent-1.1.2/c-ares” && if [ -r ares_build.h ]; then cp ares_build.h ares_build.h.orig; fi && /bin/sh ./configure CONFIG_COMMANDS= CONFIG_FILES= && cp ares_config.h ares_build.h “$OLDPWD” && mv ares_build.h.orig ares_build.h) > configure-output.txt’ in /tmp/easy_install-mB7JTK/gevent-1.1.2/build/temp.linux-x86_64-2.7/c-ares
In file included from gevent/gevent.ares.c:274:
gevent/dnshelper.c:19: warning: “PyUnicode_FromString” redefined
#define PyUnicode_FromString PyBytes_FromString

In file included from /usr/include/python2.7/Python.h:88,
from gevent/gevent.ares.c:4:
/usr/include/python2.7/unicodeobject.h:281: note: this is the location of the previous definition

define PyUnicode_FromString PyUnicodeUCS4_FromString

jcdick1 · April 9, 2019, 11:17pm

In file included from gevent/cares_ntop.h:3,
from gevent/dnshelper.c:13,
from gevent/gevent.ares.c:274:
gevent/dnshelper.c: In function âgevent_append_addrâ:
c-ares/inet_ntop.h:21:33: warning: implicit declaration of function âinet_ntopâ; did you mean âares_inet_ntopâ? [-Wimplicit-function-declaration]
#define ares_inet_ntop(w,x,y,z) inet_ntop(w,x,y,z)
^~~~~~~~~
gevent/dnshelper.c:53:9: note: in expansion of macro âares_inet_ntopâ
if (ares_inet_ntop(family, src, tmpbuf, tmpsize)) {
^~~~~~~~~~~~~~
In file included from gevent/cares_pton.h:3,
from gevent/gevent.ares.c:272:
gevent/dnshelper.c: In function âgevent_make_sockaddrâ:
c-ares/inet_net_pton.h:20:31: warning: implicit declaration of function âinet_ptonâ; did you mean âares_inet_ptonâ? [-Wimplicit-function-declaration]
#define ares_inet_pton(x,y,z) inet_pton(x,y,z)
^~~~~~~~~
gevent/dnshelper.c:146:10: note: in expansion of macro âares_inet_ptonâ
if ( ares_inet_pton(AF_INET, hostp, &((struct sockaddr_in*)sa6)->sin_addr.s_addr) > 0 ) {
^~~~~~~~~~~~~~
c-ares/ares_init.c: In function âares_dupâ:
c-ares/ares_init.c:285:17: warning: argument to âsizeofâ in âstrncpyâ call is the same expression as the source; did you mean to use the size of the destination? [-Wsizeof-pointer-memaccess]
sizeof(src->local_dev_name));
^
Linking /tmp/easy_install-mB7JTK/gevent-1.1.2/build/lib.linux-x86_64-2.7/gevent/ares.so to /tmp/easy_install-mB7JTK/gevent-1.1.2/gevent/ares.so
Linking /tmp/easy_install-mB7JTK/gevent-1.1.2/build/lib.linux-x86_64-2.7/gevent/_semaphore.so to /tmp/easy_install-mB7JTK/gevent-1.1.2/gevent/_semaphore.so
File “build/bdist.linux-x86_64/egg/gevent/_socket3.py”, line 183
def makefile(self, mode=“r”, buffering=None, *,
^
SyntaxError: invalid syntax

File “/usr/opt/rockstor-core/eggs/tmpajuNMJ/gevent-1.1.2-py2.7-linux-x86_64.egg/gevent/_socket3.py”, line 183
def makefile(self, mode=“r”, buffering=None, *,
^
SyntaxError: invalid syntax

Got gevent 1.1.2.
Getting distribution for ‘djangorestframework==3.1.1’.
warning: no previously-included files matching ‘pycache’ found under directory ‘’
warning: no previously-included files matching '.py[co]’ found under directory ‘’
Got djangorestframework 3.1.1.
Getting distribution for ‘django-ztask==0.1.5’.
zip_safe flag not set; analyzing archive contents…
Got django-ztask 0.1.5.
Getting distribution for ‘django-pipeline==1.6.9’.
warning: no files found matching '.html’ under directory ‘pipeline/jinja2’
warning: no previously-included files matching ‘.pyc’ found under directory ‘tests’
warning: no previously-included files matching '.pyo’ found under directory ‘tests’
warning: no previously-included files matching ‘’ found under directory ‘tests/node_modules’
warning: no previously-included files matching '’ found under directory ‘tests/npm-cache’
warning: no previously-included files matching ‘*’ found under directory ‘tests/npm’
Got django-pipeline 1.6.9.
Getting distribution for ‘django-oauth-toolkit==0.9.0’.
Got django-oauth-toolkit 0.9.0.
Getting distribution for ‘distribute>=0.6.35’.
warning: install_lib: ‘build/lib’ does not exist – no Python modules to install

Got distribute 0.7.3.
Getting distribution for ‘coverage’.
warning: no previously-included files found matching ‘ci/appveyor.token’
no previously-included directories found matching ‘doc/_build’
no previously-included directories found matching ‘tests/eggsrc/build’
no previously-included directories found matching ‘tests/eggsrc/dist’
no previously-included directories found matching ‘tests/eggsrc/.egg-info’
warning: no previously-included files matching '.py[co]’ found anywhere in distribution
Got coverage 4.5.3.
Getting distribution for ‘chardet==2.3.0’.
warning: no files found matching ‘COPYING’
warning: no files found matching ‘.html’ under directory ‘docs’
warning: no files found matching '.css’ under directory ‘docs’
warning: no files found matching ‘.png’ under directory ‘docs’
warning: no files found matching '.gif’ under directory ‘docs’
zip_safe flag not set; analyzing archive contents…
Got chardet 2.3.0.
Getting distribution for ‘URLObject==2.1.1’.
zip_safe flag not set; analyzing archive contents…
Got URLObject 2.1.1.
Getting distribution for ‘python-engineio>=1.0.0’.
warning: no files found matching ‘README.md’
File “build/bdist.linux-x86_64/egg/tests/asyncio/test_asyncio_server.py”, line 26
async def mock_coro(*args, **kwargs):
^
SyntaxError: invalid syntax

File “build/bdist.linux-x86_64/egg/tests/asyncio/test_asyncio_client.py”, line 32
async def mock_coro(*args, **kwargs):
^
SyntaxError: invalid syntax

File “build/bdist.linux-x86_64/egg/tests/asyncio/test_async_asgi.py”, line 20
async def mock_coro(*args, **kwargs):
^
SyntaxError: invalid syntax

File “build/bdist.linux-x86_64/egg/tests/asyncio/test_asyncio_socket.py”, line 22
async def mock_coro(*args, **kwargs):
^
SyntaxError: invalid syntax

File “build/bdist.linux-x86_64/egg/engineio/asyncio_socket.py”, line 13
async def poll(self):
^
SyntaxError: invalid syntax

File “build/bdist.linux-x86_64/egg/engineio/asyncio_client.py”, line 37
async def connect(self, url, headers={}, transports=None,
^
SyntaxError: invalid syntax

File “build/bdist.linux-x86_64/egg/engineio/asyncio_server.py”, line 66
async def send(self, sid, data, binary=None):
^
SyntaxError: invalid syntax

File “build/bdist.linux-x86_64/egg/engineio/async_drivers/aiohttp.py”, line 92
async def call(self, environ):
^
SyntaxError: invalid syntax

File “build/bdist.linux-x86_64/egg/engineio/async_drivers/asgi.py”, line 57
async def _app(receive, send):
^
SyntaxError: invalid syntax

File “build/bdist.linux-x86_64/egg/engineio/async_drivers/tornado.py”, line 28
async def get(self, *args, **kwargs):
^
SyntaxError: invalid syntax

File “build/bdist.linux-x86_64/egg/engineio/async_drivers/sanic.py”, line 37
async def read(self, length=None):
^
SyntaxError: invalid syntax

File “/usr/opt/rockstor-core/eggs/tmpr17nRx/python_engineio-3.5.1-py2.7.egg/engineio/asyncio_client.py”, line 37
async def connect(self, url, headers={}, transports=None,
^
SyntaxError: invalid syntax

File “/usr/opt/rockstor-core/eggs/tmpr17nRx/python_engineio-3.5.1-py2.7.egg/engineio/asyncio_server.py”, line 66
async def send(self, sid, data, binary=None):
^
SyntaxError: invalid syntax

File “/usr/opt/rockstor-core/eggs/tmpr17nRx/python_engineio-3.5.1-py2.7.egg/engineio/asyncio_socket.py”, line 13
async def poll(self):
^
SyntaxError: invalid syntax

File “/usr/opt/rockstor-core/eggs/tmpr17nRx/python_engineio-3.5.1-py2.7.egg/engineio/async_drivers/aiohttp.py”, line 92
async def call(self, environ):
^
SyntaxError: invalid syntax

File “/usr/opt/rockstor-core/eggs/tmpr17nRx/python_engineio-3.5.1-py2.7.egg/engineio/async_drivers/asgi.py”, line 57
async def _app(receive, send):
^
SyntaxError: invalid syntax

File “/usr/opt/rockstor-core/eggs/tmpr17nRx/python_engineio-3.5.1-py2.7.egg/engineio/async_drivers/sanic.py”, line 37
async def read(self, length=None):
^
SyntaxError: invalid syntax

File “/usr/opt/rockstor-core/eggs/tmpr17nRx/python_engineio-3.5.1-py2.7.egg/engineio/async_drivers/tornado.py”, line 28
async def get(self, *args, **kwargs):
^
SyntaxError: invalid syntax

File “/usr/opt/rockstor-core/eggs/tmpr17nRx/python_engineio-3.5.1-py2.7.egg/tests/asyncio/test_async_asgi.py”, line 20
async def mock_coro(*args, **kwargs):
^
SyntaxError: invalid syntax

File “/usr/opt/rockstor-core/eggs/tmpr17nRx/python_engineio-3.5.1-py2.7.egg/tests/asyncio/test_asyncio_client.py”, line 32
async def mock_coro(*args, **kwargs):
^
SyntaxError: invalid syntax

File “/usr/opt/rockstor-core/eggs/tmpr17nRx/python_engineio-3.5.1-py2.7.egg/tests/asyncio/test_asyncio_server.py”, line 26
async def mock_coro(*args, **kwargs):
^
SyntaxError: invalid syntax

File “/usr/opt/rockstor-core/eggs/tmpr17nRx/python_engineio-3.5.1-py2.7.egg/tests/asyncio/test_asyncio_socket.py”, line 22
async def mock_coro(*args, **kwargs):
^
SyntaxError: invalid syntax

Got python-engineio 3.5.1.
Getting distribution for ‘greenlet>=0.4.9’.
Got greenlet 0.4.15.
Getting distribution for ‘futures>=2.1.3’.
warning: no files found matching ‘CHANGES’
Got futures 3.2.0.
Getting distribution for ‘oauthlib==1.0.1’.
zip_safe flag not set; analyzing archive contents…
Got oauthlib 1.0.1.
Getting distribution for ‘django-braces>=1.2.2’.
no previously-included directories found matching ‘docs/_build’
Got django-braces 1.13.0.
Generated script ‘/usr/opt/rockstor-core/bin/django’.
Generated script ‘/usr/opt/rockstor-core/bin/test’.
Installing scripts.
Generated script ‘/usr/opt/rockstor-core/bin/rockon-json’.
Generated script ‘/usr/opt/rockstor-core/bin/backup-plugin’.
Generated script ‘/usr/opt/rockstor-core/bin/qgroup-test’.
Generated script ‘/usr/opt/rockstor-core/bin/prep_db’.
Generated script ‘/usr/opt/rockstor-core/bin/ovpn-client-gen’.
Generated script ‘/usr/opt/rockstor-core/bin/delete-api-key’.
Generated script ‘/usr/opt/rockstor-core/bin/pwreset’.
Generated script ‘/usr/opt/rockstor-core/bin/initrock’.
Generated script ‘/usr/opt/rockstor-core/bin/ovpn-initpki’.
Generated script ‘/usr/opt/rockstor-core/bin/data-collector’.
Generated script ‘/usr/opt/rockstor-core/bin/docker-wrapper’.
Generated script ‘/usr/opt/rockstor-core/bin/rcli’.
Generated script ‘/usr/opt/rockstor-core/bin/st-snapshot’.
Generated script ‘/usr/opt/rockstor-core/bin/mnt-share’.
Generated script ‘/usr/opt/rockstor-core/bin/backup-config’.
Generated script ‘/usr/opt/rockstor-core/bin/debug-mode’.
Generated script ‘/usr/opt/rockstor-core/bin/replicad’.
Generated script ‘/usr/opt/rockstor-core/bin/qgroup-maxout-limit’.
Generated script ‘/usr/opt/rockstor-core/bin/delete-rockon’.
Generated script ‘/usr/opt/rockstor-core/bin/bootstrap’.
Generated script ‘/usr/opt/rockstor-core/bin/ovpn-client-print’.
Generated script ‘/usr/opt/rockstor-core/bin/send-replica’.
Generated script ‘/usr/opt/rockstor-core/bin/st-system-power’.
Generated script ‘/usr/opt/rockstor-core/bin/flash-optimize’.
Generated script ‘/usr/opt/rockstor-core/bin/qgroup-clean’.
Generated script ‘/usr/opt/rockstor-core/bin/st-pool-scrub’.
Installing gunicorn.
Getting distribution for ‘gunicorn==19.7.1’.
warning: no previously-included files matching ‘pycache’ found under directory ‘’
warning: no previously-included files matching '.py[co]’ found under directory ‘*’
File “build/bdist.linux-x86_64/egg/gunicorn/workers/_gaiohttp.py”, line 84
yield from self.wsgi.close()
^
SyntaxError: invalid syntax

File “/usr/opt/rockstor-core/eggs/tmpK36S11/gunicorn-19.7.1-py2.7.egg/gunicorn/workers/_gaiohttp.py”, line 84
yield from self.wsgi.close()
^
SyntaxError: invalid syntax

Got gunicorn 19.7.1.
Generated script ‘/usr/opt/rockstor-core/bin/rockon-json’.
Generated script ‘/usr/opt/rockstor-core/bin/backup-plugin’.
Generated script ‘/usr/opt/rockstor-core/bin/qgroup-test’.
Generated script ‘/usr/opt/rockstor-core/bin/prep_db’.
Generated script ‘/usr/opt/rockstor-core/bin/ovpn-client-gen’.
Generated script ‘/usr/opt/rockstor-core/bin/delete-api-key’.
Generated script ‘/usr/opt/rockstor-core/bin/pwreset’.
Generated script ‘/usr/opt/rockstor-core/bin/initrock’.
Generated script ‘/usr/opt/rockstor-core/bin/ovpn-initpki’.
Generated script ‘/usr/opt/rockstor-core/bin/data-collector’.
Generated script ‘/usr/opt/rockstor-core/bin/docker-wrapper’.
Generated script ‘/usr/opt/rockstor-core/bin/rcli’.
Generated script ‘/usr/opt/rockstor-core/bin/st-snapshot’.
Generated script ‘/usr/opt/rockstor-core/bin/mnt-share’.
Generated script ‘/usr/opt/rockstor-core/bin/backup-config’.
Generated script ‘/usr/opt/rockstor-core/bin/debug-mode’.
Generated script ‘/usr/opt/rockstor-core/bin/replicad’.
Generated script ‘/usr/opt/rockstor-core/bin/qgroup-maxout-limit’.
Generated script ‘/usr/opt/rockstor-core/bin/delete-rockon’.
Generated script ‘/usr/opt/rockstor-core/bin/bootstrap’.
Generated script ‘/usr/opt/rockstor-core/bin/ovpn-client-print’.
Generated script ‘/usr/opt/rockstor-core/bin/send-replica’.
Generated script ‘/usr/opt/rockstor-core/bin/st-system-power’.
Generated script ‘/usr/opt/rockstor-core/bin/flash-optimize’.
Generated script ‘/usr/opt/rockstor-core/bin/qgroup-clean’.
Generated script ‘/usr/opt/rockstor-core/bin/st-pool-scrub’.
Generated script ‘/usr/opt/rockstor-core/bin/gunicorn’.
Generated script ‘/usr/opt/rockstor-core/bin/gunicorn_paster’.
Generated script ‘/usr/opt/rockstor-core/bin/django-admin’.
Generated script ‘/usr/opt/rockstor-core/bin/coverage2’.
Generated script ‘/usr/opt/rockstor-core/bin/coverage-2.7’.
Generated script ‘/usr/opt/rockstor-core/bin/coverage’.
Generated script ‘/usr/opt/rockstor-core/bin/chardetect’.
Generated script ‘/usr/opt/rockstor-core/bin/distro’.
Generated script ‘/usr/opt/rockstor-core/bin/django-admin.py’.
Installing init-gunicorn.
Installing nginx-conf.
Installing shellinabox-conf.
Installing stop-shellinabox.
stop-shellinabox: Running ‘systemctl stop shellinaboxd’
Failed to stop shellinaboxd.service: Unit shellinaboxd.service not loaded.
Installing django-settings-conf.
Unused options for django-settings-conf: ‘datastore’ ‘pgdata_dir’ ‘reppubport’ ‘reprecvport’ ‘repsinkport’ ‘smartdb’.
Installing test-settings-conf.
Installing mime-types.
Installing extra-stuff.
extra-stuff: Running ‘mkdir -p /usr/opt/rockstor-core/var/log &&
mkdir -p /usr/opt/rockstor-core/src/rockstor/logs &&
usermod -a -G root nginx &&
systemctl disable nginx’
Unused options for extra-stuff: ‘update-command’.
Installing js-clean.
js-clean: Running ‘rm -rf /usr/opt/rockstor-core/static/js/lib.bkp &&
mv -f /usr/opt/rockstor-core/static/js/lib /usr/opt/rockstor-core/static/js/lib.bkp’
mv: cannot stat ‘/usr/opt/rockstor-core/static/js/lib’: No such file or directory
Unused options for js-clean: ‘update-command’.
Installing js-libraries.
js-libraries: Extracting package to /usr/opt/rockstor-core/static/js/lib
Installing js-sync.
js-sync: Running ‘rsync -avz /usr/opt/rockstor-core/static/js/lib.bkp/ /usr/opt/rockstor-core/static/js/lib/’
sending incremental file list
rsync: change_dir “/usr/opt/rockstor-core/static/js/lib.bkp” failed: No such file or directory (2)

sent 20 bytes received 12 bytes 64.00 bytes/sec
total size is 0 speedup is 0.00
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1189) [sender=3.1.3]
Unused options for js-sync: ‘update-command’.
Installing collectstatic.
Installing supervisor.
Getting distribution for ‘supervisor==3.0b1’.
Got supervisor 3.0b1.
Getting distribution for ‘meld3>=0.6.5’.
zip_safe flag not set; analyzing archive contents…
Got meld3 1.0.2.
Generated script ‘/usr/opt/rockstor-core/bin/rockon-json’.
Generated script ‘/usr/opt/rockstor-core/bin/backup-plugin’.
Generated script ‘/usr/opt/rockstor-core/bin/qgroup-test’.
Generated script ‘/usr/opt/rockstor-core/bin/prep_db’.
Generated script ‘/usr/opt/rockstor-core/bin/ovpn-client-gen’.
Generated script ‘/usr/opt/rockstor-core/bin/delete-api-key’.
Generated script ‘/usr/opt/rockstor-core/bin/pwreset’.
Generated script ‘/usr/opt/rockstor-core/bin/initrock’.
Generated script ‘/usr/opt/rockstor-core/bin/ovpn-initpki’.
Generated script ‘/usr/opt/rockstor-core/bin/data-collector’.
Generated script ‘/usr/opt/rockstor-core/bin/docker-wrapper’.
Generated script ‘/usr/opt/rockstor-core/bin/rcli’.
Generated script ‘/usr/opt/rockstor-core/bin/st-snapshot’.
Generated script ‘/usr/opt/rockstor-core/bin/mnt-share’.
Generated script ‘/usr/opt/rockstor-core/bin/backup-config’.
Generated script ‘/usr/opt/rockstor-core/bin/debug-mode’.
Generated script ‘/usr/opt/rockstor-core/bin/replicad’.
Generated script ‘/usr/opt/rockstor-core/bin/qgroup-maxout-limit’.
Generated script ‘/usr/opt/rockstor-core/bin/delete-rockon’.
Generated script ‘/usr/opt/rockstor-core/bin/bootstrap’.
Generated script ‘/usr/opt/rockstor-core/bin/ovpn-client-print’.
Generated script ‘/usr/opt/rockstor-core/bin/send-replica’.
Generated script ‘/usr/opt/rockstor-core/bin/st-system-power’.
Generated script ‘/usr/opt/rockstor-core/bin/flash-optimize’.
Generated script ‘/usr/opt/rockstor-core/bin/qgroup-clean’.
Generated script ‘/usr/opt/rockstor-core/bin/st-pool-scrub’.
Generated script ‘/usr/opt/rockstor-core/bin/echo_supervisord_conf’.
Generated script ‘/usr/opt/rockstor-core/bin/pidproxy’.
Generated script ‘/usr/opt/rockstor-core/bin/supervisorctl’.
Generated script ‘/usr/opt/rockstor-core/bin/supervisord’.
Generated script ‘/usr/opt/rockstor-core/bin/django-admin’.
Generated script ‘/usr/opt/rockstor-core/bin/coverage2’.
Generated script ‘/usr/opt/rockstor-core/bin/coverage-2.7’.
Generated script ‘/usr/opt/rockstor-core/bin/coverage’.
Generated script ‘/usr/opt/rockstor-core/bin/chardetect’.
Generated script ‘/usr/opt/rockstor-core/bin/distro’.
Generated script ‘/usr/opt/rockstor-core/bin/django-admin.py’.
Installing supervisord-conf.
Unused options for supervisord-conf: ‘host’ ‘jd_cmd’ ‘port’ ‘sm_cmd’ ‘smart_manager_cmd’.
Installing create-cert.
create-cert: Running ‘if [[ ! -d /usr/opt/rockstor-core/certs ]]; then
mkdir /usr/opt/rockstor-core/certs;
openssl req -nodes -newkey rsa:2048 -keyout /usr/opt/rockstor-core/certs/first.key -out /usr/opt/rockstor-core/certs/rockstor.csr -sha256 -subj “/C=US/ST=Rockstor user’s state/L=Rockstor user’s city/O=Rockstor user/OU=Rockstor dept/CN=rockstor.user” &&
openssl rsa -in /usr/opt/rockstor-core/certs/first.key -out /usr/opt/rockstor-core/certs/rockstor.key &&
openssl x509 -in /usr/opt/rockstor-core/certs/rockstor.csr -out /usr/opt/rockstor-core/certs/rockstor.cert -sha256 -req -signkey /usr/opt/rockstor-core/certs/rockstor.key -days 3650
fi’
Generating a 2048 bit RSA private key
…+++
…+++
writing new private key to ‘/usr/opt/rockstor-core/certs/first.key’

writing RSA key
Signature ok
subject=C = US, ST = Rockstor user’s state, L = Rockstor user’s city, O = Rockstor user, OU = Rockstor dept, CN = rockstor.user
Getting Private key
Unused options for create-cert: ‘update-command’.
Installing docker-conf.
Installing rockstor-pre-systemd-conf.
Installing rockstor-systemd-conf.
Installing bootstrap-systemd-conf.
Installing setup-systemd-links.
setup-systemd-links: Running ‘cp -f /usr/opt/rockstor-core/conf/rockstor-pre.service /etc/systemd/system/rockstor-pre.service &&
systemctl enable rockstor-pre.service &&
cp -f /usr/opt/rockstor-core/conf/rockstor.service /etc/systemd/system/rockstor.service &&
systemctl enable rockstor.service &&
cp -f /usr/opt/rockstor-core/conf/rockstor-bootstrap.service /etc/systemd/system/rockstor-bootstrap.service &&
systemctl enable rockstor-bootstrap.service &&
systemctl daemon-reload’
Unused options for setup-systemd-links: ‘update-command’.
Installing def-kernel.
def-kernel: Running ‘grubby --set-default=/boot/vmlinuz-‘4.12.4-1.el7.elrepo.x86_64’’
/bin/sh: grubby: command not found
Unused options for def-kernel: ‘update-command’.
Installing start-rockstor.
start-rockstor: Running ‘systemctl stop rockstor-pre &&
systemctl start rockstor’
A dependency job for rockstor.service failed. See ‘journalctl -xe’ for details.
Unused options for start-rockstor: ‘update-command’.

Flox · April 10, 2019, 1:17am

That may be a long shot, but what about the output of journalctl -xe? I’ve identified a problem with database migrations once like that.

jcdick1 · April 10, 2019, 1:47am

Sorry for all the bold. That’s just how its being formatted copy’n’pasted from the SSH session.

linux-1ktn:/usr/opt # journalctl -xe
– Defined-By: systemd
– Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel

– An ExecStart= process belonging to unit rockstor-pre.service has exited.

– The process’ exit code is ‘exited’ and its exit status is 1.
Apr 09 17:58:11 linux-1ktn systemd[1]: rockstor-pre.service: Failed with result ‘exit-code’.
– Subject: Unit failed
– Defined-By: systemd
– Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel

– The unit rockstor-pre.service has entered the ‘failed’ state with result ‘exit-code’.
Apr 09 17:58:11 linux-1ktn systemd[1]: Failed to start Tasks required prior to starting Rockstor.
– Subject: A start job for unit rockstor-pre.service has failed
– Defined-By: systemd
– Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel

– A start job for unit rockstor-pre.service has finished with a failure.

– The job identifier is 37658 and the job result is failed.
Apr 09 17:58:11 linux-1ktn systemd[1]: Dependency failed for RockStor startup script.
– Subject: A start job for unit rockstor.service has failed
– Defined-By: systemd
– Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel

– A start job for unit rockstor.service has finished with a failure.

– The job identifier is 37657 and the job result is dependency.
Apr 09 17:58:11 linux-1ktn systemd[1]: rockstor.service: Job rockstor.service/start failed with result ‘dependency’.

Flox · April 10, 2019, 2:04am

You can try to put the log text between a pair of triple backticks.

Related to your build issues, I can think of a couple things to try, from memory:

Systemctl status -l rockstor-pre rockstor

You can also keep a look at the journal (journalctl -f) while building, that’s how I’ve identified the migration quirk I’ve mentioned above.

jcdick1 · April 20, 2019, 10:24pm

I ran the systemctl status -l rockstor-pre.service:

  systemctl status -l rockstor-pre.service
    â rockstor-pre.service - Tasks required prior to starting Rockstor
       Loaded: loaded (/etc/systemd/system/rockstor-pre.service; enabled; vendor preset: disabled)
       Active: failed (Result: exit-code) since Sat 2019-04-20 17:17:54 CDT; 3min 13s ago
      Process: 59376 ExecStart=/opt/rockstor-core/bin/initrock (code=exited, status=1/FAILURE)
     Main PID: 59376 (code=exited, status=1/FAILURE)

    Apr 20 17:17:54 homer initrock[59376]:   File "/opt/rockstor-core/bin/initrock", line 38, in <module>
    Apr 20 17:17:54 homer initrock[59376]:     sys.exit(scripts.initrock.main())
    Apr 20 17:17:54 homer initrock[59376]:   File "/opt/rockstor-core/src/rockstor/scripts/initrock.py", line 362, in main
    Apr 20 17:17:54 homer initrock[59376]:     run_command([SYSCTL, 'restart', 'postgresql'])
    Apr 20 17:17:54 homer initrock[59376]:   File "/opt/rockstor-core/src/rockstor/system/osi.py", line 121, in run_command
    Apr 20 17:17:54 homer initrock[59376]:     raise CommandException(cmd, out, err, rc)
    Apr 20 17:17:54 homer initrock[59376]: system.exceptions.CommandException: Error running a command. cmd = /usr/bin/systemctl restart >
    Apr 20 17:17:54 homer systemd[1]: rockstor-pre.service: Main process exited, code=exited, status=1/FAILURE
    Apr 20 17:17:54 homer systemd[1]: rockstor-pre.service: Failed with result 'exit-code'.
    Apr 20 17:17:54 homer systemd[1]: Failed to start Tasks required prior to starting Rockstor.

Followed by “systemctl status -l rockstor.service”:

systemctl status -l rockstor.service
â rockstor.service - RockStor startup script
   Loaded: loaded (/etc/systemd/system/rockstor.service; enabled; vendor preset: disabled)
   Active: inactive (dead)

Apr 20 17:17:54 homer systemd[1]: Dependency failed for RockStor startup script.
Apr 20 17:17:54 homer systemd[1]: rockstor.service: Job rockstor.service/start failed with result 'dependency'.

jcdick1 · April 20, 2019, 10:46pm

At this point, if your test systems are able to build, I can’t help but think there’s something you have installed that is not part of the base “server” opensuse install, and not either listed under “packages needed” in the dev notes or one of the zypper-resolved dependencies for those specific packages.

Or, “git clone https://github.com/rockstor/rockstor-core.git” doesn’t get all the source code required.

Flox · April 26, 2019, 2:00pm

Hi @jcdick1,
I finally could find some time to try on a fresh TW (20190423) install and am too getting errors. I do not get pg_config-related errors as I followed @phillxnet’s instructions re-postgresql10, but postgresql still is giving me trouble:

Upon building, I see the following error:

system.exceptions.CommandException: Error running a command. cmd = su - postgres -c /usr/bin/initdb /var/lib/pgsql/data. rc = 1. stdout = ['']. stderr = ['/var/lib/pgsql/.bash_profile: line 1: /usr/share/postgresql/bash_profile: Permission denied', 'initdb: no data directory specified', 'You must identify the directory where the data for this database system', 'will reside.  Do this with either the invocation option -D or the', 'environment variable PGDATA.', '']

(reproduced below for better formatting):

linux-6au0:/ # su - postgres -c /usr/bin/initdb /var/lib/pgsql/data
/var/lib/pgsql/.bash_profile: line 1: /usr/share/postgresql/bash_profile: Permission denied
initdb: no data directory specified
You must identify the directory where the data for this database system
will reside.  Do this with either the invocation option -D or the
environment variable PGDATA.

I do have postgresql10 selected:

linux-6au0:/ # find . -name pg_config
./usr/lib/postgresql10/bin/pg_config
./usr/bin/pg_config
./etc/alternatives/pg_config
linux-6au0:/ #
linux-6au0:/ # systemctl status -l postgresql
● postgresql.service - PostgreSQL database server
   Loaded: loaded (/usr/lib/systemd/system/postgresql.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Fri 2019-04-26 09:39:06 EDT; 19min ago
 Main PID: 1512 (code=exited, status=1/FAILURE)

Apr 26 09:38:44 linux-6au0 postgresql-script[1032]: Initializing PostgreSQL 10.7 at location /var/lib/pgsql/data
Apr 26 09:38:51 linux-6au0 postgresql-script[1032]: 2019-04-26 09:38:51.477 EDT   [1512]LOG:  listening on IPv6 address "::1", port 5432
Apr 26 09:38:51 linux-6au0 postgresql-script[1032]: 2019-04-26 09:38:51.478 EDT   [1512]LOG:  listening on IPv4 address "127.0.0.1", port 5432
Apr 26 09:38:51 linux-6au0 postgresql-script[1032]: 2019-04-26 09:38:51.480 EDT   [1512]LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
Apr 26 09:38:51 linux-6au0 postgresql-script[1032]: 2019-04-26 09:38:51.488 EDT   [1512]LOG:  listening on Unix socket "/tmp/.s.PGSQL.5432"
Apr 26 09:38:51 linux-6au0 postgresql-script[1032]: 2019-04-26 09:38:51.496 EDT   [1512]LOG:  redirecting log output to logging collector process
Apr 26 09:38:51 linux-6au0 postgresql-script[1032]: 2019-04-26 09:38:51.496 EDT   [1512]HINT:  Future log output will appear in directory "log".
Apr 26 09:38:51 linux-6au0 systemd[1]: Started PostgreSQL database server.
Apr 26 09:39:06 linux-6au0 systemd[1]: postgresql.service: Main process exited, code=exited, status=1/FAILURE
Apr 26 09:39:06 linux-6au0 systemd[1]: postgresql.service: Failed with result 'exit-code'.

I’ll try to look further and see if I missed something somewhere, or if I can fix it.

Flox · April 26, 2019, 3:21pm

Fixed!

I changed the following line:

github.com

rockstor/rockstor-core/blob/33917f112011d13e702633f525f2afdc0de78238/src/rockstor/scripts/initrock.py#L360


      
          # Conditionally run this only if found (CentOS/RedHat script)
          if os.path.isfile('/usr/bin/postgresql-setup'):
              logger.debug('running postgresql-setup initdb')
              # Legacy (CentOS) db init command
              run_command(['/usr/bin/postgresql-setup', 'initdb'])
          else:
              ## In eg openSUSE run the generic initdb from postgresql##-server
              if os.path.isfile('/usr/bin/initdb'):
                  logger.debug('running generic initdb on {}'.format(pg_data))
                  run_command(
                      ['su', '-', 'postgres', '-c', '/usr/bin/initdb', pg_data])
          logging.info('Done.')
          run_command([SYSCTL, 'restart', 'postgresql'])
          run_command([SYSCTL, 'status', 'postgresql'])
          logging.debug('Postgresql restarted')
          logging.info('Creating app databases...')
          run_command(['su', '-', 'postgres', '-c', '/usr/bin/createdb smartdb'])
          logging.debug('smartdb created')
          run_command(['su', '-', 'postgres', '-c',
                       '/usr/bin/createdb storageadmin'])
          logging.debug('storageadmin created')

from:

['su', '-', 'postgres', '-c', '/usr/bin/initdb', pg_data])

to:

['su', '-', 'postgres', '-c', '-D', '/usr/bin/initdb', pg_data])

…and then rebuilding from source.

It may break non-TW builds, though, and I’m not sure what upstream change (if any) caused this. You can try it if you are still stuck, though.

Hope this helps,

phillxnet · April 26, 2019, 8:03pm

@Flox Nice find, funny how this has now cropped up as it did work previously. Tried to find upstream changes but ran out of time.

Can’t look further right now but can chip in with PGDATA normally being set within the postgres users home directory via their:

/var/lib/pgsql/.bash_profile

which has the following:

# Get PGDATA from /etc/sysconfig/postgresql
if test -f /etc/sysconfig/postgresql; then
   eval eval $(sed -n '/^POSTGRES_DATADIR=/s//PGDATA=/p' /etc/sysconfig/postgresql)
   export PGDATA
fi

with /etc/sysconfig/postgresql containing:

## Path:           Applications/PostgreSQL
## Description:    The PostgreSQL Database System
## Type:           string()
## Default:        "~postgres/data"
## ServiceRestart: postgresql
#
# In which directory should the PostgreSQL database reside?
#
POSTGRES_DATADIR="~postgres/data"

## Path:           Applications/PostgreSQL
## Description:    The PostgreSQL Database System
## Type:           string()
## Default:        ""
## ServiceRestart: postgresql
#
# The options that are given to the PostgreSQL master daemon on startup.
# See the manual pages for postmaster and postgres for valid options.
#
# Don't put "-D datadir" here since it is set by the startup script
# based on the variable POSTGRES_DATADIR above.
#
POSTGRES_OPTIONS=""

which has remained unchanged in all my prior root snapshots in Tumbleweed.

Would be nice to track this down as in all other invocations we haven’t had to specify the data directory explicitly like this. I’d also rather stick to the default directory as with some more recent dealings with JeOS (Leap 15.0) and postgres I’ve ended up with the default being created anyway, even after I’d explicitly named a custom db directory.

And as another test I have:

su - postgres
# so we are not 'logged in as' postgres user
env | grep PGDATA
# gave the following output
PGDATA=/var/lib/pgsql/data
# and to return to root user:
exit

all with:

distro 
Name: openSUSE Tumbleweed
Version: 20190423

Strange.

@Flox and @jcdick1 do either of your problem systems have the following file:

/var/lib/pgsql/.bash_profile

If so if your could compare it’s contents as that should be what sets this required and apparently missing environment variable in the problem cases.

Thanks

I’ll try to get back to the ‘core’ development on openSUSE soon and will then do a ‘nice & clean’ build but must first complete some back-end server stuff I have to get done.

@jcdick1 Well done on persevering with this one; as you see we have ‘changing sands’ and ongoing concerns but it looks like you should now be good to go on a recent ‘clean’ build: with the indicated modifications. And there is provision for a source build to be transitioned to an rpm one once the relevant rpms are uploaded. The provision for the additional openSUSE rpm repositories is one of the server aspects I’m working on in the background currently.

@Flox Thanks for stepping up to this one, again; and nice exposition. We will have to decide what we are to do with this apparently ‘missing’ PGDATA env var situation.

Flox · April 26, 2019, 8:23pm

Thanks a lot for your input @phillxnet!

That’s actually really interesting, here’s why:

I do have this .bash_profile but it’s a simply referral to /usr/share/postgresql/bash_profile

rockdev:~ # ls -lht /var/lib/pgsql
total 12K
-rw------- 1 postgres postgres  23 Apr 26 16:11 .bash_history
drwx------ 1 postgres postgres 550 Apr 26 16:06 data
-rw-r--r-- 1 postgres postgres 859 Apr 26 11:18 initlog
-rw-r----- 1 postgres postgres  34 Apr 26 09:03 .bash_profile
rockdev:~ #
rockdev:~ # cat /var/lib/pgsql/.bash_profile
/usr/share/postgresql/bash_profile

That one, in turn, does not set PGDATA at all, hence our issue:

rockdev:~ # cat /usr/share/postgresql/bash_profile
#!/bin/bash

shopt -s nullglob

if test "$#" -ne 1; then
        echo "usage: $0 pgversion" 1>&2
        exit 1
fi

PRIO=$1
case "$PRIO" in
  postgresql*)
    PRIO="${PRIO##postgresql}"
  ;;
esac

PGBASEDIR=/usr/lib/postgresql$PRIO
if [ $PRIO -lt 80 ] ; then
  PRIO="${PRIO}0"
fi

PGBINDIR=$PGBASEDIR/bin

for FILE in $PGBINDIR/*; do
    NAME=$(basename $FILE)
    DIR=/usr/bin
    SLAVES="$SLAVES --slave $DIR/$NAME $NAME $FILE"
done

if test -n "$SLAVES"; then
    update-alternatives --quiet --install \
        /usr/lib/postgresql postgresql $PGBASEDIR $PRIO \
        $SLAVES
else
    update-alternatives --remove postgresql $PGBASEDIR
fi

Note, however, that the relevant contents of /etc/sysconfig/postgresql are the same as yours.

I’ll try and see if I have the same behavior with Leap 15.1, by curiosity.

EDIT:
As mentioned above, I get this error on a fresh TW_20190423 install, so could this discrepancy with your system be explained by the fact that yours carried over the correct files from previous versions?

phillxnet · April 27, 2019, 5:51pm

@Flox

This now very much looks to be the case, with your report. My instance of postgres user’s ‘.bash_profile’ appears to date from June 2018:

-rw-r----- 1 postgres postgres 192 Jun  9  2018 .bash_profile

So I’m guessing it was a post install script of an earlier postgresql-server rpm or the like that generated it. It is not owned by any current package. And once in place was, via policy for such things, left as was. Nice to know anyway.

Lets move to conclude this then.
Given the current case is that we can’t depend on this PGDATA environmental variable being set for us any longer and my prior code looks to have assumed it’s existence we do as you and the error message suggests and explicitly set our db data dir via the -D option to our pg_data python var which previously shadowed the same env var anyway.

Note worthy is that our related CentOS executions are also apparently relying on the same PGDATA env var but have as yet not run into this issue. But for the initialisation we have thus far used the RedHat specific script. But we should concentrate on the openSUSE side really, ie from a CentOS based Rockstor install we have:

cat /var/lib/pgsql/.bash_profile  
[ -f /etc/profile ] && source /etc/profile

PGDATA=/var/lib/pgsql/data
export PGDATA

Upon another look your prior correction to:

Looks to have the -D option in the wrong place as it’s an option to be passed to initdb not ~~the login environment~~ su:

where as the ‘su -’ is shorthand for switch user with login environment of that user; see man su:

-, -l, --login
              Start the shell as a login shell with an environment similar to a real login:

I did the following to pass the -D option to initdb:

                    ['su', '-', 'postgres', '-c', '/usr/bin/initdb -D {}'.format(pg_data)])

which may not be the most elegant but worked for me and seems to read OK: i.e. we are passing a single command string via the —command (-c) option on su.

Since you diagnosed that -D thingy do you fancy making an issue / pr and we can then link to it from our openSUSE build wiki? I’ve also recently updated the wiki with some of our other more recent findings, ie to install postgresql10 etc.

Yes, but also note the following:

su - postgres
-bash: /usr/share/postgresql/bash_profile: Permission denied

So although it no longer sets up our PGDATA it also doesn’t even get run!

So looks like they may have something a miss there but with the -D explicit fix we can work ‘as is’ which is convenient. Would be nice to report upstream but not sure where currently as still getting to grips with backlog and haven’t put the time into familiarity with the openSUSE bug reporting.

@jcdick1 & @Flox I’ve just re-run the wiki instructions, post changes, and I think we are good; as long as the indicated issues are observed, ie:

Recent pg_config ‘in path’ fix via alternative step (note posttresql10 pkg variants).
This recent -D switch thing detailed above.
The recent Docker json rename detailed in #2032 (though not build related).

Proved today with fresh install of 20190425 Tumbleweed snapshot.

@jcdick1 Would you mind changing this forum’s title to indicate it’s relevance to Tumbleweed as that has become it’s main focus, given our mutual aims here.

Hope that helps.

phillxnet · May 3, 2019, 5:02pm

@Flox and @jcdick1 Re missing PGDATA env var in recent Tumbleweed.

I’ve created the following issue:

github.com/rockstor/rockstor-core

[openSUSE] don't rely on PGDATA env var

opened 04:28PM - 03 May 19 UTC

closed 05:40PM - 20 May 19 UTC

phillxnet

Thanks to @FroggyFlox and forum member jcdick1 for highlighting this issue. Prev…iously we implicitly relied upon the PGDATA env variable. Some more recent openSUSE Tumbleweed releases (as of writing) no longer define this variable for the postgres user so it is suggested that, for our openSUSE code path only, we use initdb's -D option to explicitly define our desired Postresql 'data' directory. Linking to the relevant forum thread for exposition and context: https://forum.rockstor.com/t/build-instructions/5946/17 through to: https://forum.rockstor.com/t/build-instructions/5946/21

and associated pr:

github.com/rockstor/rockstor-core

[openSUSE] don't rely on PGDATA env var. Fixes #2041

rockstor:master ← phillxnet:2041_openSUSE_don't_rely_on_PGDATA_env_var

opened 04:46PM - 03 May 19 UTC

phillxnet

+1 -1

Explicitly define Postgresql db 'data' directory, in openSUSE code path, via ini…tdb's -D option. Use existing in code pg_data var which is set to the normal default of '/var/lib/pgsql/data' anyway. Prior use of pg_data in this code path was ineffectual. Fixes #2041 Ready for review. Please see issue text for context, including forum origin references.

based on our recent exposition in this thread.

Build instructions

define PyUnicode_FromString PyUnicodeUCS4_FromString

linux-1ktn:/usr/opt # journalctl -xe – Defined-By: systemd – Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel

– An ExecStart= process belonging to unit rockstor-pre.service has exited.

– The process’ exit code is ‘exited’ and its exit status is 1. Apr 09 17:58:11 linux-1ktn systemd[1]: rockstor-pre.service: Failed with result ‘exit-code’. – Subject: Unit failed – Defined-By: systemd – Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel

– A start job for unit rockstor-pre.service has finished with a failure.

– A start job for unit rockstor.service has finished with a failure.

linux-1ktn:/usr/opt # journalctl -xe
– Defined-By: systemd
– Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel

– The process’ exit code is ‘exited’ and its exit status is 1.
Apr 09 17:58:11 linux-1ktn systemd[1]: rockstor-pre.service: Failed with result ‘exit-code’.
– Subject: Unit failed
– Defined-By: systemd
– Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel