Unknown internal error doing a POST to /api/sm/services/shellinaboxd/start

ArmyHill01 · November 1, 2021, 4:17pm

Brief description of the problem

Unable to enable ‘Shell in a box’ service

Detailed step by step instructions to reproduce the problem

Fresh install of 4.0.8, updated to 4.0.9. Attempted to start service from service page as well as System Shell page

Web-UI screenshot

Error Traceback provided on the Web-UI

See screen shot, error window is empty

phillxnet · November 1, 2021, 6:11pm

@ArmyHill01 Hello again. Well done on getting the 4.0.8 installer built. Did you use a Leap 15.2 or Leap 15.3 profile ?

Have you rebooted since the install took place. It’s possible to use the system prior to reboot but there is something concerning the shellinabox package and it’s systemd intergration, which we fixed upstream actually, that may need a reboot. Worth a try.

Also, for diagnosis, see if you can start the service from the command line it may help with diagnosing the issue:

systemctl start shellinabox
systemctl status shellinabox

The openSUSE shellinabox was last fixed in:

Issue:

github.com/rockstor/rockstor-core

[openSUSE] shellinabox

opened 10:51PM - 12 Jan 19 UTC

closed 02:36PM - 10 Mar 20 UTC

flukejones

`shellinabox` will need to be packaged. However: 1. It's most current release… is a bit over two years old, 2. It uses openssl v1.0 (Tumbleweed defaults to 1.1) 3. Upstream doesn't include a systemd unit So I'm wondering if there are better alternatives - but these must be written in systems languages, not for eg, python or node-js, these are just far too heavy-weight. Fedora package source is [here](https://src.fedoraproject.org/cgit/rpms/shellinabox.git/tree/) At this point I also suggest creating an _official_ rockstor account on [openSUSE OBS](https://build.opensuse.org/) to ease package building and supply, you should be able to get packages merged to official channels if required also.

pull request:

github.com/rockstor/rockstor-core

[openSUSE] adapt to shellinabox package differences. Fixes #2006

rockstor:master ← phillxnet:2006_openSUSE_shellinabox

opened 07:57PM - 02 Mar 20 UTC

phillxnet

+426 -315

A recently available OBS shells.repo hosted shellinabox package had breaking dif…ferences from our prior epel CentOS pacakge. The following code changes were made to support both packages concurrently ~~(see testing section for pending changes to the OBS package)~~: Fixes #2006 - Move to re-configure service on every service enable and black format the associated file: shellinaboxd_service.py, with minor refactoring. - Remove build time 'initial' config file instantiation, along with the prior listed change this establishes a single point, within code, where our desired configuration is written. This also ensures we overwrite package default config prior to initial service start. - Establish distro specific configuration options: the proposed openSUSE shells repo package has a different certificates location. - Go with both package defaults re user/group for running the shellinabox deamon. - Account for differing default configuration file names. - Account for differing systemd service file names by normalising on our existing 'shellinaboxd' and establishing this if not found during the initrock.py script run by rockstor-pre.service. - Improve comments in prep_db.py pertaining to shellinaboxd + black format. - Remove unused status() and associated import from shell.py. - Pass systemd service name to shell.py to reduce hardcoded instances. - Move shell.py to string.format + black formatting. - Fix existing bug where shellinabox config file options had no closing quote and carriage return. @FroggyFlox Ready for review. Apologies as I didn't manage to split out all of the black formatting, but the largest file was blacked in it's own commit. Not all files were blacked but bit by bit. ## Testing: In our openSUSE targets I used a slightly modified version of the indicated OBS shell.repo shellinabox package ~~that carries two proposed, and submitted for inclusion upstream, systemd service file fixes by this pr's author: https://build.opensuse.org/package/show/home:rockstor:branches:shells/shellinabox~~ EDIT: The proposed / submitted upstream changes have now been accepted upstream. As part of this edit I have update the indicated repos to reference the upstream one that now incorporates the referenced systemd servcie file fixes. This package was chosen via research by @FroggyFlox and myself as it had recently been adopted and moved into the official OBS shells.repo repository. It also contains fixes for shortfalls of a few other existing shellinabox packages available in other individual OBS repos, ie no system, firewalld, and openssl v1.1 compatibility. All of which we require. ### Leap15.1: ``` zypper addrepo --refresh -p105 http://download.opensuse.org/repositories/shells/openSUSE_Leap_15.1/ shells ``` ### Tumbleweed: ``` zypper addrepo --refresh -p105 http://download.opensuse.org/repositories/shells/openSUSE_Tumbleweed/ shells ``` with subsequent: ``` zypper refresh (trust temporarily or always) ``` A lower priority than default (99) is suggested to allow for default priority override (edit) and to avoid other shell entries within this repo from superseding default packages. ~~if this package enters the standard repositories; although a vendor change would likely also be required.~~ Functionally tested to work as expected on a 'rockstor' CentOS base and the openSUSE Leap15.1, & Tubleweed distributions, all via a from scratch source build of rockstor-core. ## Caveates: 1. Under certain timings the initial invocation of the shell when started from the Web-UI header link and it's own page System - System Shell results in "502 Bad Gateway" To reproduce: When the service is off and is enabled via: - System - System Shell - or Web-UI header “System Shell” Work around = Web Browser Page refresh (does not work with popup window option). 2. Popup window mode flaky. When “Open shell in a popup windows” config option is selected (non default): - Works but is a little flaky as needs a master window refresh to re-enable popup window invocation once one is closed. These 2 caveats were observed in Leap15.1, and have been seen in our to-be legacy CentOS base but have yet to be seen in the Tumbleweed instance; but are believed to be generic to the base packages' function or our general implementation and were considered out side the realm of this pull request.

And on a dev machine here, Leap 15.3 base from one of our own installers, I have:

Session timed out as I was taking the screen grab.

I’m assuming here you installed from a build installer. Or did you start with a vanilla Leap 15.x base?

Let us know re the service and I’ll try and take a look at this again some soon.

Thanks again for the report. We’ve not had any other reports of this of late but I don’t think it’s that popular a feature. But I can’t actually know.

Hope that helps, at least to know the code involved. And let us know what that service start/status outputs.

phillxnet · November 1, 2021, 6:19pm

@ArmyHill01
I’ve also just tried, in the services page, enabling and disabling the shellinabox service without error. Little switch waited and then moved as expected.

What is the system you are trying this out on. If it’s very slow that could potentially cause an issue. Let us know a bit more info and that may help. We pre-install the shellinabox packge in the installer via:

github.com

rockstor/rockstor-installer/blob/master/rockstor.kiwi#L470


    <package name="python2-requests"/>  <!--Needed by api_wrapper.py-->
    <package name="python2-setuptools"/>  <!--pkg_resources supervisorctl-->
    <package name="python2-six"/>
    <package name="python3-distro"/>  <!--dnf-plugins-core dep-->
    <package name="python3-python-dateutil"/>  <!--dnf-plugins-core dep-->
    <package name="realmd"/>
    <package name="realmd-lang"/>  <!--as 'no-recommends' would skip-->
    <package name="rsync"/>
    <package name="samba"/>  <!--we miss cron via 'no-recommends'-->
    <package name="samba-winbind"/>
    <package name="shellinabox"/>  <!--for embedded Web-UI console-->
    <package name="smartmontools"/>
    <package name="systemtap-runtime"/>
    <package name="ypbind"/>
    <!--ROCKSTOR PACKAGE-->
    <!--Change to reflect the version specified, i.e. 4.0.8-0-->
    <package name="rockstor-4.0.8-0"/>
</packages>
<packages type="image" profiles="Leap15.2.RaspberryPi4">
    <package name="raspberrypi-firmware" arch="aarch64"/>
    <package name="raspberrypi-firmware-config" arch="aarch64"/>

with some earlier entries that setup the associted repo according the the profile and architecture used to build the installer.

Hope that helps and let us know how you get on. On my system here it all looks to be working as expected so it would be good to know what’s gone wrong with yours as, as always, there can always be bugs.

ArmyHill01 · November 7, 2021, 10:01pm

This was all done in a VM. I’ve yet to get v4 to install on my hardware due to a kernel panic.
The system is not slow by any means. I still have this VM if you’d like me to troubleshoot further but at this point, I’m really just trying to get an install going and setup some shares and not think about my NAS for a long while

phillxnet · November 8, 2021, 11:07am

@ArmyHill01 Hello again.
Re:

Try starting a new forum thread with all the details, i.e. hardware exhibiting this panic, version of Leap used to build the installer and the profile used, or which pre-built installer you used from our downloads page etc.

That way hopefully we can narrow down exactly what might be the cause this and hopefully resolve what’s causing it. It may be that your hardware requires a custom kernel option for instance.

Hope that helps.

ArmyHill01 · November 8, 2021, 2:47pm

Sounds good, appreciate the help!