Trouble with dropbox rockon: libdropbox_apex.so ImportError

queek · October 28, 2019, 4:14pm

I’m trying to troubleshoot some issues with the Dropbox rockon. According to my /var/log/messages there seems to be an import error for libdropbox_apex.so. Traceback:

...
Oct 28 08:56:38 ... journal: dropbox: load fq extension '/opt/dropbox/dropbox-lnx.x86_64-83.4.152/apex._apex.cpython-37m-x86_64-linux-gnu.so'
Oct 28 08:56:38 ... journal: Traceback (most recent call last):
Oct 28 08:56:38 ... journal:  File "dropbox/client/main.pyc", line 18, in <module>
Oct 28 08:56:38 ... journal:  File "dropbox/client/features/catalina_migration/catalina_migration_controller.pyc", line 19, in <module>
Oct 28 08:56:38 ... journal:  File "dropbox/client/features/catalina_migration/catalina_account_context.pyc", line 13, in <module>
Oct 28 08:56:38 ... journal:  File "dropbox/client/features/catalina_migration/alert_dialog.pyc", line 10, in <module>
Oct 28 08:56:38 ... journal:  File "dropbox/client/features/file_locking/base_file_locking_alert.pyc", line 14, in <module>
Oct 28 08:56:38 ... journal:  File "dropbox/client/features/legacy_ui_launcher.pyc", line 21, in <module>
Oct 28 08:56:38 ... journal:  File "dropbox/client/configuration/manager.pyc", line 45, in <module>
Oct 28 08:56:38 ... journal:  File "dropbox/client/configuration/utils.pyc", line 18, in <module>
Oct 28 08:56:38 ... journal:  File "dropbox/client/autorestart.pyc", line 33, in <module>
Oct 28 08:56:38 ... journal:  File "dropbox/foundation/html_views/common/__init__.pyc", line 9, in <module>
Oct 28 08:56:38 ... journal:  File "dropbox/foundation/html_views/common/content_metadata.pyc", line 18, in <module>
Oct 28 08:56:38 ... journal:  File "dropbox/foundation/html_views/common/interface.pyc", line 18, in <module>
Oct 28 08:56:38 ... journal:  File "dropbox/foundation/context/__init__.pyc", line 14, in <module>
Oct 28 08:56:38 ... journal:  File "apex/__init__.pyc", line 6, in <module>
Oct 28 08:56:38 ... journal:  File "<_bootstrap_overrides>", line 153, in load_module
Oct 28 08:56:38 ... journal: ImportError: libdropbox_apex.so: cannot open shared object file: No such file or directory
Oct 28 08:56:38 ... journal: !! dropbox: fatal python exception:

Over on manjaro.org there is a solution posted for this problem but I’m not sure how to implement it on the rockon. I have no experience with Docker so if anyone can suggest things to try I would be very appreciative.

(I have tried uninstalling and reinstalling the rock on and I get the same error. My guess is that the files are persistent.)

Flox · October 28, 2019, 5:56pm

Hi @queek, and welcome!

Nice tracking of the error and a potential fix, that will probably speed up fixing your problem.

I’m not a Dropbox user but I just tried to install it on Rockstor 3.9.2-50 and didn’t get your error:

Checking for latest Dropbox version...
Latest   : 83.4.152
Installed: 83.4.152
Dropbox is up-to-date
Starting dropboxd (83.4.152)...
dropbox: locating interpreter
!! dropbox: failed to create log file (Permission denied)!
dropbox: initializing
dropbox: initializing python 3.7.2
dropbox: setting program path '/opt/dropbox/dropbox-lnx.x86_64-83.4.152/dropbox'
dropbox: setting home path '/opt/dropbox/dropbox-lnx.x86_64-83.4.152'
dropbox: setting python path '/opt/dropbox/dropbox-lnx.x86_64-83.4.152:/opt/dropbox/dropbox-lnx.x86_64-83.4.152/python-packages-37.zip'
dropbox: python initialized
dropbox: running dropbox
dropbox: setting args
dropbox: applying overrides
dropbox: running main script
dropbox: load fq extension '/opt/dropbox/dropbox-lnx.x86_64-83.4.152/cryptography.hazmat.bindings._constant_time.cpython-37m-x86_64-linux-gnu.so'
dropbox: load fq extension '/opt/dropbox/dropbox-lnx.x86_64-83.4.152/cryptography.hazmat.bindings._openssl.cpython-37m-x86_64-linux-gnu.so'
dropbox: load fq extension '/opt/dropbox/dropbox-lnx.x86_64-83.4.152/cryptography.hazmat.bindings._padding.cpython-37m-x86_64-linux-gnu.so'
dropbox: load fq extension '/opt/dropbox/dropbox-lnx.x86_64-83.4.152/psutil._psutil_linux.cpython-37m-x86_64-linux-gnu.so'
dropbox: load fq extension '/opt/dropbox/dropbox-lnx.x86_64-83.4.152/psutil._psutil_posix.cpython-37m-x86_64-linux-gnu.so'
dropbox: load fq extension '/opt/dropbox/dropbox-lnx.x86_64-83.4.152/linuxffi.pthread._linuxffi_pthread.cpython-37m-x86_64-linux-gnu.so'
dropbox: load fq extension '/opt/dropbox/dropbox-lnx.x86_64-83.4.152/cpuid.compiled._cpuid.cpython-37m-x86_64-linux-gnu.so'
dropbox: load fq extension '/opt/dropbox/dropbox-lnx.x86_64-83.4.152/apex._apex.cpython-37m-x86_64-linux-gnu.so'
dropbox: load fq extension '/opt/dropbox/dropbox-lnx.x86_64-83.4.152/tornado.speedups.cpython-37m-x86_64-linux-gnu.so'
dropbox: load fq extension '/opt/dropbox/dropbox-lnx.x86_64-83.4.152/linuxffi.resolv.compiled._linuxffi_resolv.cpython-37m-x86_64-linux-gnu.so'
dropbox: load fq extension '/opt/dropbox/dropbox-lnx.x86_64-83.4.152/librsyncffi.compiled._librsyncffi.cpython-37m-x86_64-linux-gnu.so'
dropbox: load fq extension '/opt/dropbox/dropbox-lnx.x86_64-83.4.152/linuxffi.sys.compiled._linuxffi_sys.cpython-37m-x86_64-linux-gnu.so'
dropbox: load fq extension '/opt/dropbox/dropbox-lnx.x86_64-83.4.152/posixffi.libc._posixffi_libc.cpython-37m-x86_64-linux-gnu.so'
dropbox: load fq extension '/opt/dropbox/dropbox-lnx.x86_64-83.4.152/linuxffi.gnu.compiled._linuxffi_gnu.cpython-37m-x86_64-linux-gnu.so'
This computer isn't linked to any Dropbox account...
(...)

This was after a restart, so it explains why you don’t see Dropbox updating itself to the latest version (already up-to-date).

That being said, it is surprising that I’m not able to reproduce that error, especially considering the fact that you’ve found other references to it from other users of this container. I actually just found another reference to it on the Github repository of the docker image itself. Fortunately, this one also includes a fix, but unfortunately it hasn’t been merged yet:

Where the problem lies, however, is that this fix relates to files that are persistent (/opt/ folder) so that fix may need to be ran every time the rock-on restarts. Note also that I’m not sure whether it actually does fix the issue as I’m not able to test it (because I do not see the error to begin with). This also means that it is possible to run this fix only if the container is running… and because I don’t know whether that is the case, I’m not sure you can even test it. If you’re willing to try, you can:

docker exec dropbox sh -c "chmod a+rx /opt/dropbox/dropbox-lnx*/*.so"

This simply fixes the permission on the files that cause problem.

Note, however, that I would be curious to know which version of Dropbox is running in the rock-on in your machine as it may have been fixed by Dropbox in a more recent version. You should be able to see this by monitoring the container’s log in real time while starting the rock-on:

docker inspect -f dropbox

and then toggle the rock-on ON from the webUI. Alternatively, you can also run the command below, assuming the rock-on is running:

docker exec dropbox sh -c "cat /opt/dropbox/VERSION"

I apologize for the slightly misconstructed answer… I’m now running a bit short on time, unfortunately, but I do hope this helps. Let us know how it goes!

Thanks!

queek · October 28, 2019, 7:20pm

Thanks for looking into this @Flox.

It appears that my Dropbox is stuck in a perpetual restart loop (it tries to initialize, encounters the ImportError, stops for a minute, then restarts itself. That means that every time I try to do a docker exec dropbox sh -c ... command I get the response

Error response from daemon: Container 997...d31 is restarting, wait until the container is running

The docker inspect command you posted doesn’t have a specified format string so I’m not sure exactly which parameter you’re looking for.

According to the /var/log/messages log that the Dropbox docker uses, the installed version is 83.4.152 (latest: 83.4.152, “Dropbox is up-to-date”). Looking back through that log, it looks like on my last system restart a week ago Dropbox updated itself from 78.4.119 to 83.4.152 at which point it started breaking. Perhaps my issue is arising from the fact that I updated rather than fresh-installed like you did. Do you know if there’s a way to rollback the installed Dropbox (or do a clean install)?

Flox · October 28, 2019, 9:26pm

That’s what I was worried about .

Sounds good, that’s the version I got on my test install…

My apologies, this command should have indeed read docker inspect dropbox.

Based on the fact that this issue seems to have been present on a lot of users, I would tend to direct my eyes to the way Dropbox tends to update itself…

Regarding a clean install, absolutely, that’s a good idea to try. Because you mentioned in your original post that you already tried uninstalling/reinstalling it, I will assume that uninstalling it is not a problem for you.
NOTE, however, that I am not familiar with how Dropbox syncs between the local content and what is on “the cloud side”, so I am hoping–but do not know for sure–that it will not simply remove everything that is on the cloud side thinking the empty local folder you will use as a test is what your Dropbox account should look like. That would be a very dangerous way to behave from Dropbox, but again, I am not familiar with it so I do not know. On my tests, the dropbox container keeps complaining that “the computer isn’t linked to any account” when it first boots, which would indicate it is safe, but I’m not sure how the “computer” is identified. I’m probably being extra careful here, but I’d rather not make you destroy all the data you have stored in your Dropbox.

One first step (optional, because it may be overkill) would be to make sure no “ghost” container remains. If the rock-on uninstall from Rockstor webUI went well, the two commands below should fail.

docker stop dropbox
docker rm dropbox

Then, you could try the “cleanest” option and simply create 2 new shares (to make sure that no file from your previous install is carried over), make sure the user with choose during the Dropbox rock-on install has permissions on them, and then install the dropbox rock-on using these two new shares. When I need to monitor a docker container’s start procedure, I usually like to have a terminal open at the same time and follow the logs using journalctl -f… you should see the entire startup process and it is what I posted in my first message above.

This should start without a problem…

Let’s see how it goes from here…

Hope this helps,

queek · October 28, 2019, 11:48pm

I’ve tried a number of things now with no success.

updated Rocstor from 3.9.2-48 to -50
created new shares (testing both user/group options of root as owner or dropbox as owner)
setting the skip updating flag to True

Each time the same error is raised on first start, so something must be persisting for me to see this error while you do not.

The skip update flag appears to always be ignored and Dropbox auto updates from 11.4.21 to 83.4.152 each time the rock on is reinstalled. A closer inspection of the Dockerfile suggests that Dropbox will download its own binaries, so the flag may not actually have any effect. Given that the janeczku/docker-dropbox Github repository was last updated 3 years ago, I don’t think that the fix will be merged any time soon. That leaves trying to incorporate the changed line from the merge request into the rock on that’s installed.

I have no experience with Docker, so I don’t know is there’s an easy way to go about this. Perhaps the easiest would be for someone to build a new docker based on the fixed code and change the link included in the rock on to pull from the new docker build.

I’m open to other suggestions as well, preferably not involving wiping and reinstalling Rockstor itself (I’ll draw the line before that).

P.S. the restriction of the number of links a new user can post makes it impossible to include all the relevant links. Is that restriction really necessary?

Flox · October 29, 2019, 1:54pm

In this case, you can try to re-download the docker image and see how it goes… It’s worth a try, at least. After uninstalling the dropbox rock-on:

/opt/rockstor/bin/delete-rockon dropbox.
This will remove all references of the dropbox rock-on from Rockstor’s database and will also proceed wit the removal of the docker container, and the docker image. None of your data in shares will be touched. The output should be:

Rock-On(dropbox) metadata in the db is deleted

Verify that any trace of your previous dropbox rock-on and its underlying docker container is gone:
a. docker ps -a: lists all your docker containers, running or stopped. You shouldn’t see any mention of the dropbox container. If you do, first stop it with docker stop <container name> and then remove it with docker rm <container name> (the container name is taken from the output of docker ps -a).
b. docker images: lists all docker images present on your machine. As for the container, you shouldn’t see any mention of dropbox there. If you do, remove the image with docker rmi <image ID> (the image ID is taken from the output of docker images).
As we made new shares just for testing this (as in your point 2 in your most recent post), we can delete their content. Make sure to do this on your test shares that you just created as this will delete all data in them: rm -Rvf /mnt2/<dropbox-config share name>/* and rm -Rvf /mnt2/<dropbox folder share name>/*.
Back in Rockstor’s webUI, go to the Rock-ons page, and click on the Update button in the top right. This will update the list of available rock-ons.
Proceed with the installation using new shares as detailed in my previous post. For instance, on my machine I have:

Resource type	Name	Mapped representation
Share	drop-conf	/dbox/.dropbox
Share	drop-test	/dbox/Dropbox
Env	DBOX_UID	1000
Env	DBOX_GID	1000
Env	$DBOX_SKIP_UPDATE	False

Note: 1000 is the GID and UID of the Rockstor user that I set as owner of the two shares drop-conf and drop-test.

Before clicking Submit, which starts the installation process, monitor the logs in a terminal session to see what is happening with journalctl -f. On my machine I have the following:

Oct 29 06:25:36 rockdev dockerd[3894]: Checking for latest Dropbox version...
Oct 29 06:25:38 rockdev dockerd[3894]: Latest   : 83.4.152
Oct 29 06:25:38 rockdev dockerd[3894]: Installed: 11.4.21
Oct 29 06:25:38 rockdev dockerd[3894]: Downloading Dropbox v83.4.152...
Oct 29 06:25:48 rockdev dockerd[3894]: [4.8K blob data]
Oct 29 06:25:48 rockdev dockerd[3894]: Installing new version...
Oct 29 06:25:48 rockdev dockerd[3894]: Dropbox updated to v83.4.152
Oct 29 06:25:48 rockdev dockerd[3894]: Starting dropboxd (83.4.152)...
Oct 29 06:25:48 rockdev dockerd[3894]: dropbox: locating interpreter
Oct 29 06:25:48 rockdev dockerd[3894]: !! dropbox: failed to create log file (Permission denied)!
Oct 29 06:25:48 rockdev dockerd[3894]: dropbox: initializing
Oct 29 06:25:48 rockdev dockerd[3894]: dropbox: initializing python 3.7.2
Oct 29 06:25:48 rockdev dockerd[3894]: dropbox: setting program path '/opt/dropbox/dropbox-lnx.x86_64-83.4.152/dropbox'
Oct 29 06:25:48 rockdev dockerd[3894]: dropbox: setting home path '/opt/dropbox/dropbox-lnx.x86_64-83.4.152'
Oct 29 06:25:48 rockdev dockerd[3894]: dropbox: setting python path '/opt/dropbox/dropbox-lnx.x86_64-83.4.152:/opt/dropbox/dropbox-lnx.x86_64-83.4.152/python-packages-37.zip'
Oct 29 06:25:48 rockdev dockerd[3894]: dropbox: python initialized
Oct 29 06:25:48 rockdev dockerd[3894]: dropbox: running dropbox
Oct 29 06:25:48 rockdev dockerd[3894]: dropbox: setting args
Oct 29 06:25:48 rockdev dockerd[3894]: dropbox: applying overrides
Oct 29 06:25:48 rockdev dockerd[3894]: dropbox: running main script
Oct 29 06:25:48 rockdev dockerd[3894]: dropbox: load fq extension '/opt/dropbox/dropbox-lnx.x86_64-83.4.152/cryptography.hazmat.bindings._constant_time.cpython-37m-x86_64-linux-gnu.so'
Oct 29 06:25:49 rockdev dockerd[3894]: dropbox: load fq extension '/opt/dropbox/dropbox-lnx.x86_64-83.4.152/cryptography.hazmat.bindings._openssl.cpython-37m-x86_64-linux-gnu.so'
Oct 29 06:25:49 rockdev dockerd[3894]: dropbox: load fq extension '/opt/dropbox/dropbox-lnx.x86_64-83.4.152/cryptography.hazmat.bindings._padding.cpython-37m-x86_64-linux-gnu.so'
Oct 29 06:25:49 rockdev dockerd[3894]: dropbox: load fq extension '/opt/dropbox/dropbox-lnx.x86_64-83.4.152/psutil._psutil_linux.cpython-37m-x86_64-linux-gnu.so'
Oct 29 06:25:49 rockdev dockerd[3894]: dropbox: load fq extension '/opt/dropbox/dropbox-lnx.x86_64-83.4.152/psutil._psutil_posix.cpython-37m-x86_64-linux-gnu.so'
Oct 29 06:25:49 rockdev dockerd[3894]: dropbox: load fq extension '/opt/dropbox/dropbox-lnx.x86_64-83.4.152/linuxffi.pthread._linuxffi_pthread.cpython-37m-x86_64-linux-gnu.so'
Oct 29 06:25:50 rockdev dockerd[3894]: dropbox: load fq extension '/opt/dropbox/dropbox-lnx.x86_64-83.4.152/cpuid.compiled._cpuid.cpython-37m-x86_64-linux-gnu.so'
Oct 29 06:25:50 rockdev dockerd[3894]: dropbox: load fq extension '/opt/dropbox/dropbox-lnx.x86_64-83.4.152/apex._apex.cpython-37m-x86_64-linux-gnu.so'
Oct 29 06:25:50 rockdev dockerd[3894]: dropbox: load fq extension '/opt/dropbox/dropbox-lnx.x86_64-83.4.152/tornado.speedups.cpython-37m-x86_64-linux-gnu.so'
Oct 29 06:25:50 rockdev dockerd[3894]: dropbox: load fq extension '/opt/dropbox/dropbox-lnx.x86_64-83.4.152/linuxffi.resolv.compiled._linuxffi_resolv.cpython-37m-x86_64-linux-gnu.so'
Oct 29 06:25:50 rockdev dockerd[3894]: dropbox: load fq extension '/opt/dropbox/dropbox-lnx.x86_64-83.4.152/librsyncffi.compiled._librsyncffi.cpython-37m-x86_64-linux-gnu.so'
Oct 29 06:25:51 rockdev dockerd[3894]: dropbox: load fq extension '/opt/dropbox/dropbox-lnx.x86_64-83.4.152/linuxffi.sys.compiled._linuxffi_sys.cpython-37m-x86_64-linux-gnu.so'
Oct 29 06:25:51 rockdev dockerd[3894]: dropbox: load fq extension '/opt/dropbox/dropbox-lnx.x86_64-83.4.152/posixffi.libc._posixffi_libc.cpython-37m-x86_64-linux-gnu.so'
Oct 29 06:25:51 rockdev dockerd[3894]: dropbox: load fq extension '/opt/dropbox/dropbox-lnx.x86_64-83.4.152/linuxffi.gnu.compiled._linuxffi_gnu.cpython-37m-x86_64-linux-gnu.so'
Oct 29 06:25:57 rockdev dockerd[3894]: This computer isn't linked to any Dropbox account...

Let’s see how that goes.

You are correct, this flag doesn’t seem to have any effect, thanks for pointing it out. After looking into the docker image, this flag is simply used to skip the check for update if present, regardless of its value. I’ll look in more details into why it is ignored when I have some time as I suspect this has something to do with its particular syntax (the $ symbol is particular is not that common in what I’ve seen in docker ENV variables and this may throw off Rockstor).

I am afraid so too. As this change is in the docker container itself, there is unfortunately no way I can think of for now for us to incorporate this into the rock-on.

You shouldn’t have to do that, indeed, especially because this issue is very unlikely to be related to Rocsktor itself.

Let us know how that goes and what you get and then we can adjust.

queek · October 29, 2019, 6:03pm

I followed the steps you outlined to the letter and I still get the same error. I had been avoiding wiping the dropbox share contents (~40 GB) but tried that as a last resort, still yielding the same error.

As a last last resort I tried installing the Dropbox rockon using the root UID/GID and it successfully installed. So… perhaps there’s a permissions passthrough problem? According to the web UI both shares are owned by the UID/GID I was using so it should have had sufficient permissions. I’m not overly familiar with permissions and restrictions, so I’m not sure how to interpret this. I’m assuming that using the root user to run a rockon is non-ideal, but I’ll leave it like this until we can get the root cause sorted out.