Hi,
I’m very new to Rockstor and Centos. Just trying to get my head around everything.
I’m wondering if anyone knows if it’s possible to run a Postgresql 9.5 server on a Rockstor 3.9 system? It wouldn’t have to be visible from the Rockstor Web UI or to the Rockons.
An application I use implements collaborative project sharing via Postgresql 9.5. So far I have been able to install the current repository for Postgreql in the Rockstor shell but attempting to yum install postgresql95 returns an error message:
It appears to be stuck on existing PGDG repositories in the system that are no longer available.
My attempts to remove the existing repositories appear to have been unsuccessful.
I have also tried to access the existing Postgresql installed in Rockstor which I believe is version 9.2, however as I don’t know the default password for the installed postgres user I can’t access it.
I don’t have much experience, but if you were to run multiple versions at the same time, I think you would have to take some precautions. another option could be to use a docker version of the Postgresql (even better, if you created a RockOn for that ) from here: https://hub.docker.com/_/postgres
You could of course also check whether there is a dockerized version of your collaboration app that contains everything needed to run it.
Hi @Hooverdan,
Thank you so much for your reply.
I think PostgresQL is available in a Docker version already. Unfortunately I don’t have enough knowledge on these things to adapt the what I know to the usage of a Docker version. I attempted the Docker installation for Postgresql and it didn’t appear in the web UI for Rockstor so I suspect I would need to go looking for it and its configuration files.
It’s possible that DaVinci Resolve (the application I’m referring to that uses Postgresql 9.5) will be backwards compatible with 9.2 but I believe that to establish a database for DaVinci Resolve in the existing Rockstor Postgresql I will need to know the postgres user account password. So far I haven’t been successful in finding out what that password is.
As a side-note, after further investigating, I now realise that Rockstor itself uses Postgresql as its database back-end so error the messages I was getting make more sense. I still feel that ‘under the hood’ I should be able to install Postgresql 9.5 using a different install / data path and independent configuration files. After all, this is still Centos 7.
I really appreciate your response. Thank you again.
That seemed to force the repository to be installed. I then had to:
#vi /etc/yum.repos.d
and remove all repository references that did not relate to postgresql 9.5. I then modified that base URL of the reference to be exactly what the postgresql.org website links to. After that I tried to:
yum install postgresql95 postgresql95-server
and the installation worked.
I have also been able to
systemctl enable postgresql-9.5
and then also
systemctl start postgresql-9.5
once that /var/lib/pgsql/9.5/data/pg_hba.conf was modified so that Postgresql 9.5 was listening on port 5431 instead of 5432.
At this point I think the only thing stopping me from connecting to the database is the fact that I don’t know what the default password is for the postgres user in Rocktor. Once I know that I will be able to create another super user for the PSQL shell and then I believe I should have a working Postgresql 9.5 on Rockstor.
To first approach your other recent questions, in part:
Rockstor rather expects to have postgresql all to itself as the entire Rockstor system is build with an appliance angle to it. And as such on occasions we simply wipe the entirety of all databases known to the server, hence advising against using this appliance based instance of postgress
The only one I know, setup by setup is:
user: “rocky”
pass: “rocky”
Accessible only from the local host.
But please read the rest of this post as I strongly believe you are going about this whole endeavour wrongly.
What you really need however is to not use this postgresql instance but to establish one of your own.
I would suggest modifying the following proposed postgresql Rock-on to your requirements.
i.e. changing the tag to your required postgresql version. That pr is almost ready from memory and it may be that if you find a variant of it useful you could submit that as a far far safer way to stand up a truely indipendant database entirely for your own use. It’s data share would then be managed by Rockstor also which will make things much easier for you re snapshots etc.
But better still, as @Hooverdan suggests you should really be looking to finding a docker container that serves your DaVinci Resolve requirement, ideally with a build in db. You plan to manage this db along side Rockstor’s one is likely to add a great deal of fragility to your setup. Especially if you don’t take full management from here on in of it. And given you indicate not being well up on such things this is unwise.
Essentially we have such things in the code as:
With the most pertinent bit being:
shutil.rmtree("/var/lib/pgsql/data")
I.e. a forced wipe of all known databases. Hence the advise here trying to guard you against this possibility by suggesting a Rock-on.
And if you then make this db instance accessible from other than localhost then all will potentially have access to your Rockstor database. DB’s are complex stuff with particularly complex access rights. If you are not expert it’s likely your time is better spend on creating a purpose build Rock-on. And the one I linked to above is possibly just that. And if you wish to try and use that one as a base but modified to your needs (tag change) then the following might be of use:
So in short I would strongly advise against using this ‘native’ postgresql instance and run one within a Rock-on. Also all future support enquieries for this instance are at risk of being tainted by your then unknown custom db state.
Multiple co-existing Postgresql instances are just not that simple. For instance when I tried to request a different db path one time it was just ignored by ‘some’ facility and not by others. You are going to have to be very aware of this kind of behaviour with multiple simultaneous instances. There may also be access and config interferrence. This kind of requirement is exactly what docker was invented for. Which brings me to your point:
Yes, we do not surface all dockers. But we do surface all our community maintained Rock-ons, which are one json file away once you know your docker container.
Also note that we only support custom tags, which you will need, in our Stable channel currently. As from version 3.9.2-49 and was added by @Flox:
EDIT: Just realise that in this case we also need a fix that was released in 3.9.2-51 also:
That fixes dealing with multiple tags of the same docker image:
Allow multiple versions for a docker image. Fixes #2017@FroggyFlox
[END EDIT]
So have a go at testing our that proposed postgresql rock-on and modifying your copy to use the specific version you require. You may find folks here on the forum that are able to help you through this process also. That is a far more Rockstor way to achieve what you are after. Otherwise you might as well setup your own independent linux install as we are far more of an appliance really and tend to reach out beyond the normal boundaries of a regular ‘app’ in order to achieve the system configuration that our Web-UI offers.
But I say best option all around is to find a DaVinci Resolve docker image and create the single json file to make this a Rock-on. Many forum members have now done this so you should receive some help along the way hopefully. Far easier than what you are attempting to do and far far safer.
Just my opinion however and your install is obviously yours to do as you please with. but I just wanted to warn you that Rockstor is more than capable of wiping all your databases in their entirety if you do what you are currently on route to do. Go the Rock-on way and you can have as many differently versioned versions of whatever all co-existing within their own little docker worlds and all co-existing happily within Rockstor’s managed storage solution.
Hi @phillxnet,
Thank you so much for your detailed reply and all the ideas and options you have presented here.
DaVinci Resolve is an entirely proprietary application, protected by a USB licencing key. It is a video/sound editing, colour-grading media production application that is extremely complex in itself and highly reliant on having direct driver access to graphics, audio and capture/playback hardware. Hence, I don’t believe a Docker container for Resolve would be a feasible solution. Also, being a media production application, it’s really for a workstation rather than a ‘headless’ server.
The only server-based aspect of DaVinci Resolve is the Postgresql 9.5 database project format. Resolve can save its projects in its own ‘disk-based’ formats or it can create databased projects for team-sharing. Its project manager acts as a client to the Postgresql 9.5 server and has very little control over the database parameters. It can create a new database but only inside the one pre-configured Postgresql 9.5 server connection.
Before I attempted this - and it’s only testing on a completely expendable ‘server’ - I had a look at the possibility of running two different versions of Postgresql in a system. I found quite a lot of pages and post confirming that it works fine but warning that the data-path and listening ports need to be different. I also fairly quickly realised that Rockstor is based on Postgresql and that trying to remove/reinstall its own configured database would most likely incapacitate it and make it a very peculiar installation of Centos 7.
Fortunately my experiment yesterday worked and I was able to install and active Postgresql 9.5 on my Rocktor installation. The datapath for 9.5 is independent of Rockstor’s 9.2 and I have used the 9.5 postgresql.conf and pg_hba.conf to set it to listen on a different port (leaving Rockstor to operate on the default 5432 port) Both versions show as “Active” and listening on their own ports when checked by
systemctl status postgresql.service
and
systemctl status postgresql-9.5.service
It would seem at this point, my only barrier is the fact that I cannot log into the PSQL shell because I don’t know the password for the ‘postgres’ user. I’ve just attempted to log into the psql shell using the ‘rocky’ password you suggested but sadly it doesn’t work.
I will certainly go with your suggestion of trying to create a Postgresql 9.5 database inside a docker container. I suspect I may come up against the same limitation even in that instance. Because Rockstor comes with a pre-configured ‘postgres’ user which I cannot access, I am unable to log into the psql shell to create a new user account for DaVinci Resolve to access its own database through. If that were possible I would then happily manage databases over the network via DBeaver or pgAdmin4.
Probably my best alternative, overall, is to setup a separate computer to act as the project server for DaVinci Resolve. I simply wanted to see if it was possible to add the server to Rockstor. (I believe DaVinci Resolve users are in fact installing their Postgresql 9.5 databases onto their QNap NAS boxes in a similar way to what I’m attempting here).
I will do some more research to determine if there is a way to install a Postgresql 9.5 server with an entirely different admin username as the default. I will also go through everything you have suggested here that I think might work for me. Or alternatively if I can find out the default postgres password for Rockstor then that will help me along.
I appreciate your concern about the stability and feasibility of this approach and I would certainly only use it after a lot of cautious testing. Again, @phillxnet, thank you so much for your time and help.
@AudioDan24, glad to see it at least somewhat worked. I read up on the DaVinci thing and realized that the Database is only a collaboration component of that product. If you want to give the RockOn approach whirl, here are a few ideas: I created a json file and testing it on my system it seems to work very well. It allows you to pick a specific version (like e.g. 9.5.21) and allows you to map a port. I also included the SuperUser creation (along with password for this).
{
"PostgreSQL": {
"description": "PostgreSQL, Note: This rock-on requires Rockstor version 3.9.2-51 or later.",
"version":"1.2",
"website":"https://www.postgresql.org",
"more_info": "Conf.file: <code>/var/lib/pgsql/data/pg_hba.conf</code></p><p>Database:.<code>/var/lib/pgsql/data</code></p> <p>Logs: <code>/var/log/postgresql</code></p>",
"volume_add_support":true,
"containers": {
"postgresql": {
"image": "postgres",
"tag": "9.5",
"launch_order":1,
"ports": {
"5432": {
"description":"PostgreSQL port. Suggested default: 5432",
"host_default":5432,
"label":"PostgreSQL port",
"protocol":"tcp"
}
},
"volumes": {
"/var/lib/postgresql/data":{
"description":"Choose a share where the database should be stored. E.g.: create a share called postgresql-share1 for this purpose alone. ",
"label":"Data Storage"
}
},
"environment": {
"VERSION": {
"description": "Choose version of PostgreSQL: Unless you know which version to try, just type latest",
"label": "VERSION"
},
"POSTGRES_USER": {
"description": "Choose A Superuser name for the PostgreSQL installation",
"label": "SuperUser"
},
"POSTGRES_PASSWORD": {
"description": "set a password for the SuperUser.",
"label": "SuperUser password"
}
}
}
}
}
}
Put this into a json text file and place it into the local RockOn store:
/opt/rockstor/rockons-metastore
Then hit the Update button in the RockOn Screen of the WebUI. Now the PostgresSQL should show up in the list and you can install it. In the example below you can see the ports mapped differently so the RockStor DB is not affected:
In order to look inside the docker container to make changes to the config files you probably already know to use: docker exec -it <mycontainer> bash
and you should be able to connect to that Postgres instance with a string something like this (when you’re on the RockStore box, aka localhost):
psql -h localhost -p 5432 -U postgres -W
Now, I am no expert on all of this, but I essentially ignored the password request and then was in as a postgres user, and could check out with \l or \c the databases.
The rest I leave to you, as you probably are more of an expert.
I know, I shouldn’t advertise the competition, but here was a tutorial written for qnap on how to use the container and connect from an external machine that has DaVinci installed:
And with that, my science experiment is complete
Let us know, if you actually try out this approach …
Do you fancy editing the tag though as this has the same tag as the referenced pending pull request, and @AudioDan24 looks to need a 9.5 variant. Haven’t checked on exact tag required though.
@AudioDan24 Thanks for the detailed explanation, I’m get it a little more now, and quite an interesting use as it goes. And in which case @Hooverdan’s latest proposal is pretty much exactly what you need, once the tag is adjusted according to what is available (in docker hub) and your requirements. This is looking very doable now.
@Hooverdan if you fancy submitting your Rock-on json as pull request but with an appropriate tag for @AudioDan24’s requirement and some description change to indicate the set version we might get this one in quickly, especially if @AudioDan24 can test it as ‘works for me’ type thing. The prior long standing 10.6 Rock-on request has lacked any real testing, although @Flox and I have been nudging it along and now has only one outstanding issue with regard to an extraneous “host_default” entry @Flox has request be removed. If you look to that pr and model yours on the suggestions there, which I suspect you have done, we can pop this one in quick ready for @AudioDan24 testing.
Thanks all for this enlightening discussion with regard to the utility of a stand alone Postgresql Rock-on. Would be great if we had both the 10.6 and 9.5 versions in, just need to alter the descriptions to be clear to folk what they are installing and indicate there is no Web-UI button function for such a Rock-on.
@phillxnet. I left the tag alone, because from the pull request on this RockOn I was under the impression that a tag is needed for the sake of RockStor, so it doesn’t get confused, e.g. if OwnCloud is installed (since it also has a Postgres container).
In my posted version I was allowing the Postgres version to be freely selected. I assume, I can’t update the tag via the RockOn configuration screen either. Maybe @Flox can also shed some light on that: What’s the difference between the tag in the json file and a label that I can assign after the fact?
I would be more than happy to create a pull request and we can continue the discussion there, if that’s better suited.
This literally the best help I’ve received on any forum ever! Incredible!
Please excuse my minimal knowledge, I am only just managing to grasp what you are suggesting.
I’m embarrassed to say that on a previous installation of Rockstor I was able to access the the file structure from the root level via one of the types of file sharing, however, for some reason I can’t do that on this current installation. I’ve tried various share exports one service at a time but no luck. (trying to log in as ‘root’ via the file sharing login prompt give an error message.
I thought @Hooverdan instead of browsing to the folder you mentioned I would vi edit the file at the location to create it and then paste the code you gave me. However when I tried to locate the folder you listed it wasn’t there in the folder structure. I got as far as /opt/rockstor and that folder contained a bunch of files and folders that weren’t the one you mentioned above. Should I create the folder you suggested?
I’m very keen to try this out! I must admit I have some concerns about the responsiveness of DaVinci Resolve if its project is hosted inside the Docker container. I don’t know much about it but it would seem to be another ‘layer’ for the database read/writes to have to navigate through. However, I won’t know until I try it.
Please let me know about that folder location if you can. Again, sorry about my rudimentary knowledge here. I only really started working with Centos in earnest about two months ago (having been a Windows / Mac user before that).
Yes it’s not there by default and needs to be created. See:
which could be improved re the requirement to create this directory.
Docker actually has very little overhead system wise. The db should be near enough native speed.
Docker, put very simply, is kind of like a name space within the host kernel, such that the client ‘image’ believes it’s all on it’s own system. Others will likely have better explanations of it but that a quick attempt.
Hope that helps and make sure to change the tag version and remember the bit about needing Rockstor version of 3.9.2-51, and to be sure you can execute:
yum info rockstor
as the root user that you setup during install.
@Hooverdan and @AudioDan24 I had said 3.9.2-49 before but there was another bugfix, again provided by @Flox, that is need for this one. See the following pull request where we request this to be put on that pull request:
You might prefer to use “nano”, it’s pre-installed and a lot easier than vi/vim.
You could use the Web-UI embedded terminal via System → System Shell. And depending on the setting you choose (in Services - Shell In A Box, spanner icon) you may have to login as your Web-UI user first then ‘su’ to get to the root user. If you get a 502 choose the ssh which will allow direct root login. This has been improved of late as well. Just turn the service off when you are done with it to improve security.
Also if @Hooverdan hasn’t had the change make sure to edit the tag version.
Hope that helps, @Hooverdan may be able to fill in the blanks here as I’m just popping in again on this one.
Thank you @Hooverdan. I’ve attempted to create the .json file at the location:
/opt/rockstor/rockons-metadata
using the Shell In A Box interface is pretty slow because of the lack of direct copy/paste function however I pasted the data you sent me and edited it so that it sat exactly the same in the json file with every line break in the correct place. I then saved and exited the edit and re-opened the new file to confirm that it was there. (it was).
I then went and updated the RockOns list and nothing relating to Postgresql has appeared in my list.
One thing I haven’t confirmed with you yet: the folder location you gave me included an uppercase ‘R’ in “Rockstor”
/opt/Rockstor/rockons-metadata
whereas I used the existing /opt/rockstor/ folder with the lowercase ‘r’
I have now also tested the process creating a new file path that uses the “Rockstor” folder and after updating the Rockons list I still have no results
Unfortunately - again, due to my lack of knowledge - most of what @phillxnet wrote has gone over my head. I’m currently running Rockstor 3.9.1 and I’m not sure how to get to 3.9.2-49. I have activated the automatic updates feature but it tells me that at version 3.9.1 it is up-to-date.
I’m not quite sure where to go from here. I’d definitely like to test this method you’ve created and given what you found about the QNAP system it certainly seems like it might be viable. I just don’t have the knowledge to get there at the moment. I’m very much copy&paste with something of a top-level overview and not a real understanding of what I’m doing.
You’ve been incredibly helpful though. I appreciate the time and effort you’ve put in here.
Thank you both @Hooverdan and @phillxnet
@AudioDan24
I’m very sorry, I guess, I was typing (and thinking about metadata) and not copying from my test instance. The correct path should be: /opt/rockstor/rockons-metastore
And you’re right, the rockstor portion is lowercase.
I corrected it all in my original post now. My apologies again. This is probably the reason why your refresh didn’t show anything …
Ah! Thank you @Hooverdan I’ve now created this new file path and placed the .json file there.
Updating has still had no effect but I now realise it might be to do with the fact that I can’t update from Rockstor 3.9.1
I may have done something this is stopping the system from updating. If I install a clean version of Rockstor and then immediately activate automatic updates should it update to 3.9.2-55 that I see on Sourceforge? Or is it a manual installation process?
@Hooverdan Sorry I seem to have missed this earlier post of your.
My mistake I’d originally said we need at least version -49 but there was another bug, which you are referencing here, where multiple different versions of a docker image for different Rock-ons caused the tag to be ignored. Sorted by @Flox now in -51. I’ve corrected my original post.
That would be great. Thanks. As it does appear we have a need for this older Postgresql version and it makes sense to offer the version requested by @AudioDan24 as they can then test that pull request also. I’d read all of that pull request for the 10.6 version though as bar one last change I think that one is pretty much ready. I may just end up popping that one in and applying the requested change myself. So lets take these in a pull request in the rockon-registry, if required. Easier to test what you have proposed also via the previously referenced Wiki entry.
Thanks again for stepping up to this one. I rather dropped the ball on the outstanding v10.6 postgresql Rock-on but we can get your version in pretty quick hopefully based on the work in that one. And I can then circle back around to sorting the v10.6 one by merging and then applying @Flox last outstanding request there.
Cheers, and again apologies for missing this earlier post of yours.
@AudioDan24 I can hopefully address this one quickly:
Our Update Channels documentation section should help here. And if you still have a question then it may help with us improving that doc section.
The newer versions of Rockstor have a few more links to those docs so they should be easier to find once our new installer is released.
As to 3.9.2-55 it will hopefully be released in the Stable channel very soon. The sourceforge references if for the source code which has now been tagged for that version and upon tagging it gets auto zipped and uploaded. The rpm releases within the Stable Channel are announced here:
for our current 3.9.2 run. But 3.9.2-55 doesn’t contain any relevant fixes for this discussion:
Hi @phillxnet,
Thank you for explaining this. When I went to the updates page on in the Rockstor Web UI I saw the Testing Updates button was red and so without thinking about it I just assumed it wasn’t available. I’ve now activated it and I’m currently updating to the most recent version.
I’ll then go through the process that @Hooverdan has laid out for me and see what I come up with. I’m excited to give this all a test.
Thank you very much! I’ll get back to you with more information when I’ve been able to get this going.
As you are currently on our CentOS based offering the testing channel is unfortunately no longer updated. So that only gets you 4 month worth of updates on from the ISO leaving the Rockstor package at 3.9.1-16 which is still 2 year 4 months out of date. But the rest of the system will have updated. On from there for a couple of years we only did Stable Channel releases but that has change recently as we move towards being openSUSE based. The docs link I posted in my last post references this:
And the other link I referenced in that same post further details the background to this. And it doesn’t help that our Web-UI in these older versions failed to adequately link back to our docs either. So yes it does, back there, indicate that you would be running latest. But you are not. This however has also been improved upon and the language and links are now much better so in our next ISO release this should all be much clearer. These language and link improvements were mainly added:
by me in 3.9.2-52 via:
improve/add update channel/appman related text and doc links. Fixes #2106@phillxnet
With the latter addressing a similarly raised ‘latest’ misleading text issue raised by forum member @ednt.
So by the time we are done with our linux distro re-base (CentOS to openSUSE) we will again have dual Stable and Testing Channels active on a single distro base. But we are not quite there yet.
So in short you actually need to subscribe to the Stable Channel updates, but as you have already subscribed to testing you will now have been affected by a bug there where you can no long return to Stable effectively (ISO releases were based on Stable Channel releases); although it will appear to have worked it will not actually have installed the stated version.
So in short you need to read the following short forum post on moving back to Stable from testing.
This does not apply if you went straight to Stable from the ISO install as then you are remaining on more curated releases.
All a bit higgledy-piggledy I know but we are now very close to a re-launch of sorts where things will again become very simple. We are towards the end of a 3 year transition with all the associated baggage.
Hope that helps and if there is a problem with Subscribing to the Stable Channel then please contact me via PM (Private Message) here on the forum and I can hopefully get you sorted so we can move forward with sorting out this Postgresql 9.5 Rock-on that will hopefully fulfil you specific need.
Hi @phillxnet,
Sorry I haven’t been able to reply till now and thanks again for the information.
I’ve kind of lost my reference point here though. Not sure where or how to start. The website links me to the ISO image on SourceForge and that I get. Burn or restore to a USB stick and install. The 3.9.2-55 version is something else. I’m not sure how to go about installing this.
I guess for me, because I’ve spent the last three months immersing myself in Centos Linux for the soul purpose of creating a Linux-based file and project sharing system (preferably both with VPN or remote networking capabilities) for DaVinci Resolve. Centos, because that’s the only Linux distro that DaVinci Resolve officially supports.
Rockstor appealed to me because the Centos processes were becoming slightly familiar. The fact that I was able to get a Postgresql 9.5 server running along-side the Roscktor one and not interfere means I was on the right track to some degree.
If this means a shift across to an openSUSE based system then I feel like I might be going off on a tangent that’s taking away from what I’m trying to achieve. I did manage to get the Centos Rockstor to be visible via SFTP over the web which should serve our purposes well enough as a shared file server. Ideally I wanted to get Owncloud working but so far I haven’t been able to get it to connect remotely, only inside of my LAN.
The addition of the Postgresql9.5 server to this system was really just an experiment but I’m quite happy to set up a separate computer to handle it and potentially avoid undue risk to the stability of what will could become colossal databases (in the course of cutting feature-length films).
I’m very out of my depth with all of this and I might have to wait until a more consolidated version is released before I can try testing it. I also attempted to update this Centos-based version of Rockstor a few times and each time it ended with a kernel panic that stopped the system from working.
Thanks for all your help and sorry I can’t be of more assistance to you here.
) from here:
