@nigel Thanks for the report and initial investigation. That’s a funny one.
Take a look at the following forum thread where @Tony_Cristiano also had what may have been a similar or related issue with postgresql not being available:
There @Flox went for the force re-install approach and after @Tony_Cristiano then re-established the enable default and started the service it seems to have sorted itself.
Your issue may not be related but take a look to see as we have yet to get to the bottom on what happened in @Tony_Cristiano reported incident.
Yeah I tried all of that except the re-install @phillxnet, but it didn’t work. I managed to get PG working by changing it to use /tmp instead of /var/run/postgres, but then rockstor doesn’t work, as it’s coded to expect /var/run, not tmp.
@phillxnet Unfortunately this didn’t work. Looking further, it seems that the creation of the directory /run/postgresql from /usr/lib/tmpfiles.d/postgresql.conf is not being triggered for some reason…
$ cat /usr/lib/tmpfiles.d/postgresql.conf
# For the PostgreSQL server’s unix domain socket
d /run/postgresql 1775 postgres postgres - -
@phillxnet I think the problem is the tmpfiles service is failing, which is a pre-condition for PG…
$ systemctl list-units --failed
UNIT LOAD ACTIVE SUB DESCRIPTION
● dmraid-activation.service loaded failed failed Activation of DM RAID sets
● docker.service loaded failed failed Docker Application Container Engine
● postgresql.service loaded failed failed PostgreSQL database server
● rockstor-pre.service loaded failed failed Tasks required prior to starting Rockstor
● snapper-cleanup.service loaded failed failed Daily Cleanup of Snapper Snapshots
● snapper-timeline.service loaded failed failed Timeline of Snapper Snapshots
● systemd-tmpfiles-setup-dev.service loaded failed failed Create Static Device Nodes in /dev
● systemd-tmpfiles-setup.service loaded failed failed Create Volatile Files and Directories
● twonky.service loaded failed failed Twonky Server Service
LOAD = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB = The low-level unit activation state, values depend on unit type.
9 loaded units listed. Pass --all to see loaded but inactive units, too.
To show all installed unit files use ‘systemctl list-unit-files’.
:/opt/rockstor/var/log # systemctl status systemd-tmpfiles-setup
● systemd-tmpfiles-setup.service - Create Volatile Files and Directories
Loaded: loaded (/usr/lib/systemd/system/systemd-tmpfiles-setup.service; static; vendor preset: disabled)
Active: failed (Result: exit-code) since Mon 2021-04-19 16:27:35 BST; 1h 2min ago
Docs: man:tmpfiles.d(5)
man:systemd-tmpfiles(8)
Main PID: 568 (code=exited, status=1/FAILURE)
Apr 19 16:27:35 systemd-tmpfiles[568]: Unsafe symlinks encountered in /var/log/wtmp, refusing.
Apr 19 16:27:35 systemd-tmpfiles[568]: Unsafe symlinks encountered in /var/log/btmp, refusing.
Apr 19 16:27:35 systemd-tmpfiles[568]: Unsafe symlinks encountered in /var/log/lastlog, refusing.
Apr 19 16:27:35 systemd-tmpfiles[568]: Unsafe symlinks encountered in /run/nologin, refusing.
Apr 19 16:27:35 systemd-tmpfiles[568]: Unsafe symlinks encountered in /run/user, refusing.
Apr 19 16:27:35 systemd-tmpfiles[568]: Unsafe symlinks encountered in /run/utmp, refusing.
Apr 19 16:27:35 systemd[1]: systemd-tmpfiles-setup.service: Main process exited, code=exited, status=1/FAILURE
Apr 19 16:27:35 systemd[1]: Failed to start Create Volatile Files and Directories.
Apr 19 16:27:35 systemd[1]: systemd-tmpfiles-setup.service: Unit entered failed state.
Apr 19 16:27:35 systemd[1]: systemd-tmpfiles-setup.service: Failed with result ‘exit-code’.
:/opt/rockstor/var/log # systemctl status systemd-tmpfiles-setup-dev
● systemd-tmpfiles-setup-dev.service - Create Static Device Nodes in /dev
Loaded: loaded (/usr/lib/systemd/system/systemd-tmpfiles-setup-dev.service; static; vendor preset: disabled)
Active: failed (Result: exit-code) since Mon 2021-04-19 16:27:34 BST; 1h 2min ago
Docs: man:tmpfiles.d(5)
man:systemd-tmpfiles(8)
Main PID: 464 (code=exited, status=1/FAILURE)
Apr 19 16:27:34 systemd-tmpfiles[464]: Unsafe symlinks encountered in /dev/btrfs-control, refusing.
Apr 19 16:27:34 systemd-tmpfiles[464]: Unsafe symlinks encountered in /dev/loop-control, refusing.
Apr 19 16:27:34 systemd-tmpfiles[464]: Unsafe symlinks encountered in /dev/net, refusing.
Apr 19 16:27:34 systemd-tmpfiles[464]: Unsafe symlinks encountered in /dev/net/tun, refusing.
Apr 19 16:27:34 systemd-tmpfiles[464]: Unsafe symlinks encountered in /dev/ppp, refusing.
Apr 19 16:27:34 systemd-tmpfiles[464]: Unsafe symlinks encountered in /dev/userio, refusing.
Apr 19 16:27:34 systemd[1]: systemd-tmpfiles-setup-dev.service: Main process exited, code=exited, status=1/FAILURE
Apr 19 16:27:34 systemd[1]: Failed to start Create Static Device Nodes in /dev.
Apr 19 16:27:34 systemd[1]: systemd-tmpfiles-setup-dev.service: Unit entered failed state.
Apr 19 16:27:34 systemd[1]: systemd-tmpfiles-setup-dev.service: Failed with result ‘exit-code’.
Wow. Looks like / ownership got changed. Saw this…
chowned / to root, and rebooted. PG and Rockstor back up.
Now I have to figure out what did this, because it sure can’t have been Rockstor. It was owned by a share user, so must have been something happened internally.