Help recovering Web UI from crash after share changes

Troubleshooting
1

Hi all,

I’ve been following Rockstor for a few months now and really like where it’s headed so I used it for a new home server a few days ago. On a fresh install of the latest stable release, I created two shares for export via Samba. These were originally set up as Read-only access but I changed my mind and edited the shares to allow write access. Although the shares status showed the change, I was never able to actually get write permission when trying through file explorer.

I turned off/on Samba sharing to see if it would enable the write access (this proved to be a terrible idea) and almost immediately got an error message in the Web UI, first on the Samba page and then quickly cascading to every page. I used the Web UI to do a reboot of the machine, which was successful and I can log in from the command prompt, but the Web UI now will only show the Rockstor header with “Page Not Found - Sorry, an unexpected internal error has occured.” I can also still access the shares through file explorer though, although the read-only permission has still not changed.

Installation went smoothly, as did adding Rockons for Plex and OwnCloud with separate shares for each, but my Linux experience is just a hair north of zero at this point so, short of reinstalling everything again, I don’t have many ideas of where to even start on fixing this. I’ve downloaded the error log files, both from the original Samba page error and the Web UI login page error, if someone can please point me in the direction of pulling out useful information for resolving this.

I have two, 3TB drives set up in Raid 1 with Rockstor installed on a separate SSD/PCI combo I bought through the Rockstor shop.

Any help or ideas would be appreciated.

Geoff

2

@vancouverish First off welcome to the Rockstor community. Sorry to hear of you’re rather disappointing early problems, initially nothing springs to mind and your setup is pretty standard / basic as well!!.

Something rather strange has happened as although still a little buggy Rockstor is not normally as fragile as your experience has been. I’m pretty sure the Samba off/on is a red hearing though, most likely there was some background process that tripped things up.

So when in doubt - more info. If you are game or just curious then there may well be some pointers as to what has happened at the end of the /opt/rockstor/var/log/rockstor.log log file which is the main logging file for Rockstors stuff.

You can watch the new entries of this file as they happen with the following command:

tail -n 20 -f /opt/rockstor/var/log/rockstor.log

Ctrl+c to exit the tail command.

This is most conveniently done via a ssh terminal as then if there is anything of interest you can cut and paste the results here ready for comment / explanation. If on a linux or osx desktop you can use the terminal app and the command line ssh option (via “ssh root@rockstor-ip”) or if on a MS windows desktop you can install PuTTY to gain ssh client abilities.

Another file of interest is /var/log/messages.

To view a non live (but recent) version of a file you can also do the following:

less /var/log/messages

This will start at the beginning of the file and space bar moves down a page while the b key moves up a page. The g key goes to the beginning and capitalized G goes to the end, q key to exit.

These logs, especially the Rockstor log, are intended to be generally accessible so it’s definitely worth a look.

And yet another log file of interest is /var/log/yum.log which records the package manager’s update events.

less /var/log/yum.log

Also note that the rockstor.log file in your downloaded reports is definitely worth a look also. Just unzip / untar them and browse the contents, you desktop OS may respond to just double clicking on them to browse their contents.

I apologize if this info is too basic but in this context it’s tricky to assess what the north of zero level is :relaxed:.

Hope that helps as a starter.

3

@phillxnet thanks for the quick reply and pointers, very helpful. I wasn’t entirely sure what I’d be looking for in the logs but after poking around I’m wondering if there’s an issue with OwnCloud causing the problems. In that /var/log/messages file there were what must have been thousands of lines over several days, all showing some variation on:

Mar 6 23:41:22 Rockstor docker/f43fe415612b[10359]: chown: changing ownership of ‘/var/www/owncloud/data/btrfs/subvolumes/f43fe415612b2e7004601b72a881f02295
a238f7cf8c554a3f1317c646c64a23/proc/fs/nfsd’: Operation not permitted

In the error log I downloaded, I also noticed piles of issues similar to the one below:

line 164, in connect
conn = _connect(dsn, connection_factory=connection_factory, async=async)
OperationalError: FATAL: sorry, too many clients already

[08/Mar/2016 20:55:31] ERROR [smart_manager.data_collector:342] Failed to update share state… exception: Internal Server Error: No JSON object could be decoded
[08/Mar/2016 20:55:31] ERROR [smart_manager.data_collector:342] Failed to update share state… exception: Internal Server Error: No JSON object could be decoded
[08/Mar/2016 20:55:31] ERROR [storageadmin.middleware:35] Exception occured while processing a request. Path: /api/commands/refresh-share-state method: POST
[08/Mar/2016 20:55:31] ERROR [storageadmin.middleware:36] FATAL: sorry, too many clients already
Traceback (most recent call last):
File “/opt/rockstor/eggs/Django-1.6.11-py2.7.egg/django/core/handlers/base.py”,

Not sure if either of these pieces are helpful in figuring out what the issue is. If there’s anything in particular that I should be looking for then I can go back and check logs again too.

Thanks,

Geoff

4

@vancouverish Sorry for the slow response. OK, looks like you found the cause of the WebUI issue in the too many clients already bit.
It seems to be along the lines of a known issue that has affecting some users:-
Please see the forum post / thread by @Nick :-
Fatal error: Too many clients error when connecting with https after a while
@suman and @henfri have already (independantly) opened issues with one referenced in that post. The other is marked as a duplicate so it’s definately reported.
Apparently the work around for the time being is the following command

systemctl restart rockstor

Let us know how you get along, note also @joethe3rd comment in that same forum thread on a trigger for this situation.

Hope that helps.

5

Thanks, @phillxnet, that fixed it. Similar to @joethe3rd, I had the UI window open for a few days so I guess I’ll just have to be sure to close things down every day until the bug gets resolved. Just purchased a 5yr subscription to the stable stream so I get all the updates.