This is a screenshot of the Pools Site
I have changed the Quotas from “Enabled” to “Disabled” and back to “Enabled” on both Pools.
Actually a process called “data-collector” is running
I remember that this process was running immediately after the update. But I rebooted the system and I assume the process was not finished.
Process is still running…
Thanks for the additional information.
The fact that you don’t see any Rockstor and distribution information in the top right corner may be indicative that something went wrong with the update. You could try to look at the yum log (/var/log/yum.log if my memory is correct), or run yum check-update and see if it finds something available to update.
Alternatively, this could be related to the slugginess of your system caused by data-collector. As far as I know, it is used by the dashboard activity widgets (related to the fact that your widgets were all blank) and can sometimes cause this kind of issue on systems with relatively low specs. Do you still see it using that much CPU if you close Rockstor’s webUI (after waiting for a little bit to see if it calms down)?
With regards to Rockstor’s version, it could also be worth verifying whether the non-display in the UI is related to slugginess issue, or yum updates issues. You could thus make sure that the latest is indeed installed:
yum info rockstor
Do you still see it using that much CPU if you close Rockstor’s webUI (after waiting for a little bit to see if it calms down)?
Nothing changed when I left the Web GUI. Process still running with 95% CPU time.
Meanwhile I rebooted the system again.
No change on the Web GUI, no data-collector.
@Flox Re:
quote=“Flox, post:4, topic:6406”]
If you didn’t change anything, it means you probably had them enabled as this is the default setting. It is thus surprising to see them disabled for now. @phillxnet, would you have an idea as to why quotas would have been toggled?
[/quote]
There was an issue when we first switched to docker-ce where it would disable quotas. It caused us quite a bit of grief as at the time Rockstor was quite broken by a quotas disabled state, still in in testing channel. But our Stable channel releases were fixed fairly soon their after as we made a push to finally supporting a quotas disabled state. Although as you say, and as is mentioned in the tooltip, it ends up with shares showing as 0 bytes when they are disabled but a lot of btrfs stuff is much quicker. Although the speed cost of quotas is something the btrfs folks have made major improvements in recently.
I think the quotas call in this case is a red herring. It may be that, in the course of the update, which would likely have included a docker-ce update along with it, the quotas were disabled assuming a rock-ons-root share is on the affected pool.
@upapi_rockstor and @Flox the blank Web-UI header is something we have seen before and is associated with the data-collector running rampant, it’s what collects the live data for the header. So blank header and near 100% CPU on datacollector = very deleayed Web-UI as it’s waiting for non co-operative data-collector to supply data and as the non co-operative data-collector is also chewing through CPU it’s a double impact on Web-UI performance.
Now as to why. I’ve only seen this with some transient versions of python-engineio, and it’s something I’ve been looking out for whenever we roll an RPM. But given other’s haven’t experienced this it’s rather strange.
The following is the issue we keep open for this where Luke-Nukem on Github (sorry Luke I’m forgetting your forum name) ran into this when building from source. Others have too. We occasionally get version mismatches between libraries and, as I say, have to look out for this whenever we build an RPM, but given we have had no other reports with this rpm and it’s usually something that either works or not and all libraries are included in the rpm I’m at a bit of a loss. Anyway for background here is the issue. Maybe @Flox this will spark an idea. My initial thought is, as per @Flox suggestion that we have some left over code some how but I dont’ see how. Maybe browser cache as I have seen malfunctions from that during an upgrade. I.e. the browser holds onto old code / js display libraries while the rpm duly updates the backend.
https://github.com/rockstor/rockstor-core/issues/2007
In that issue their is mention of disk / pool scan optimisations, they have now been done and are actually in this release. But of course in your case are completely obfuscated by the header / data-collector issue.
And the suggested pr fix suggested in that issue, where Luke-Nukem was compiling form source, ended up being redundant as a newer version of python-engineion ended up comming out shortly their after.
Linking to a similar report from a year ago for context, although in that case I suspected insufficient cpu resource but I wasn’t sure.
And this was a much older version of Rockstor.
Sorry not to have contributed much to this issue but hopefully their is a little context that might help others help. My browser cache idea however doesn’t sit with @upapi_rockstor report of having this with 2 different browsers. @upapi_rockstor what browsers are you seeing this behaviour with, would also be good to have the browser version numbers. Bit of a shot in the dark but trying to narrow this down. I’ve not seen this on my installs here and we haven’t, yet, had another report of this in 3.9.2-49.
As @Flox suggested, if you have not browser open on the Rockstor Web-UI then their should be no data-collector running on rockstor. But as soon as you open it will trigger and of course then mess thing up in your case. If folks have any ideas, work arounds for this please chip in. Maybe we need to confirm the version of python-engineio running for example.
Hope that helps.
@phillxnet thanks for the background details. Appreciate it.
As for now I performed a system restore to the previous version 3.9.2-48. Dashboard shows all informations and the Pool page shows quota is enabled. But I see still these errors in rockstor.log
[03/Oct/2019 23:44:26] ERROR [system.osi:119] non-zero code(1) returned by command: [‘/usr/sbin/btrfs’, ‘qgroup’, ‘show’, ‘/mnt2/DATA1/rock-ons-root/btrfs/subvolumes/9423f176166cb93478baf2744612772f24f5b520e682a5697fcf624d98b6644e’]. output: [‘’] error: [“ERROR: can’t list qgroups: quotas not enabled”, ‘’]
Second attempt: successful
After restoring to the previous version the daily cron does the work:
systemctl daemon-reload done
The following updates will be applied on nas:
================================================================================
Package Arch Version Repository Size
================================================================================
Updating:
rockstor x86_64 3.9.2-49 Rockstor-Stable 17 MTransaction Summary
================================================================================
Upgrade 1 Package
The updates were successfully applied
And no Error in rockstor.log.
There might be an issue when attempting the update on the Web GUI on my system?
Nevertheless it works like a charm.
Thank you for trying to help me and Hats off for your work
Perhaps this output I’ve found in my mails when attempting the update on the Web GUI may help you finding some issues:
Loaded plugins: changelog, fastestmirror Loading mirror speeds from cached hostfile
- base: mirror1.hs-esslingen.de
- epel: mirror.nextlayer.at
- extras: mirror1.hs-esslingen.de
- updates: mirror1.hs-esslingen.de
Resolving Dependencies
–> Running transaction check
—> Package rockstor.x86_64 0:3.9.2-48 will be updated Package
—> rockstor.x86_64 0:3.9.2-49 will be an update
–> Finished Dependency ResolutionDependencies Resolved
================================================================================
Package Arch Version Repository SizeUpdating:
rockstor x86_64 3.9.2-49 Rockstor-Stable 17 MTransaction Summary
Upgrade 1 Package
Total download size: 17 M
Downloading packages:
Delta RPMs disabled because /usr/bin/applydeltarpm not installed.
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Updating : rockstor-3.9.2-49.x86_64 1/2
systemctl daemon-reload done
Cleanup : rockstor-3.9.2-48.x86_64 2/2
warning: file /opt/rockstor/eggs/zc.buildout-2.13.1-py2.7.egg/zc/buildout/tests.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/zc.buildout-2.13.1-py2.7.egg/zc/buildout/testrecipes.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/zc.buildout-2.13.1-py2.7.egg/zc/buildout/testing.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/zc.buildout-2.13.1-py2.7.egg/zc/buildout/rmtree.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/zc.buildout-2.13.1-py2.7.egg/zc/buildout/pep425tags.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/zc.buildout-2.13.1-py2.7.egg/zc/buildout/easy_install.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/zc.buildout-2.13.1-py2.7.egg/zc/buildout/download.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/zc.buildout-2.13.1-py2.7.egg/zc/buildout/configparser.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/zc.buildout-2.13.1-py2.7.egg/zc/buildout/buildout.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/zc.buildout-2.13.1-py2.7.egg/zc/buildout/init.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/zc.buildout-2.13.1-py2.7.egg/zc/init.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/windows_support.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/wheel.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/version.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/unicode_utils.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/ssl_support.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/site-patch.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/sandbox.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/py33compat.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/py31compat.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/py27compat.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/pep425tags.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/package_index.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/namespaces.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/msvc.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/monkey.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/lib2to3_ex.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/launch.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/glob.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/glibc.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/extern/init.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/extension.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/dist.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/depends.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/dep_util.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/config.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/command/upload_docs.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/command/upload.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/command/test.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/command/setopt.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/command/sdist.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/command/saveopts.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/command/rotate.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/command/register.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/command/py36compat.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/command/install_scripts.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/command/install_lib.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/command/install_egg_info.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/command/install.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/command/egg_info.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/command/easy_install.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/command/dist_info.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/command/develop.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/command/build_py.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/command/build_ext.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/command/build_clib.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/command/bdist_wininst.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/command/bdist_rpm.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/command/bdist_egg.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/command/alias.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/command/init.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/build_meta.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/archive_util.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/_vendor/six.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/_vendor/pyparsing.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/_vendor/packaging/version.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/_vendor/packaging/utils.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/_vendor/packaging/specifiers.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/_vendor/packaging/requirements.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/_vendor/packaging/markers.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/_vendor/packaging/_structures.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/_vendor/packaging/_compat.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/_vendor/packaging/init.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/_vendor/packaging/about.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/_vendor/init.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/_deprecation_warning.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/setuptools/init.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/pkg_resources/py31compat.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/pkg_resources/extern/init.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/pkg_resources/_vendor/six.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/pkg_resources/_vendor/pyparsing.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/pkg_resources/_vendor/packaging/version.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/pkg_resources/_vendor/packaging/utils.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/pkg_resources/_vendor/packaging/specifiers.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/pkg_resources/_vendor/packaging/requirements.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/pkg_resources/_vendor/packaging/markers.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/pkg_resources/_vendor/packaging/_structures.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/pkg_resources/_vendor/packaging/_compat.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/pkg_resources/_vendor/packaging/init.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/pkg_resources/_vendor/packaging/about.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/pkg_resources/_vendor/appdirs.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/pkg_resources/_vendor/init.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/pkg_resources/init.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/setuptools-40.8.0-py2.7.egg/easy_install.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/python_engineio-3.4.3-py2.7.egg/tests/common/test_socket.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/python_engineio-3.4.3-py2.7.egg/tests/common/test_server.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/python_engineio-3.4.3-py2.7.egg/tests/common/test_payload.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/python_engineio-3.4.3-py2.7.egg/tests/common/test_packet.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/python_engineio-3.4.3-py2.7.egg/tests/common/test_middleware.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/python_engineio-3.4.3-py2.7.egg/tests/common/test_client.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/python_engineio-3.4.3-py2.7.egg/tests/common/test_async_eventlet.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/python_engineio-3.4.3-py2.7.egg/tests/common/init.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/python_engineio-3.4.3-py2.7.egg/tests/asyncio/test_async_tornado.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/python_engineio-3.4.3-py2.7.egg/tests/asyncio/test_async_aiohttp.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/python_engineio-3.4.3-py2.7.egg/tests/asyncio/init.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/python_engineio-3.4.3-py2.7.egg/tests/init.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/python_engineio-3.4.3-py2.7.egg/engineio/socket.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/python_engineio-3.4.3-py2.7.egg/engineio/server.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/python_engineio-3.4.3-py2.7.egg/engineio/payload.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/python_engineio-3.4.3-py2.7.egg/engineio/packet.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/python_engineio-3.4.3-py2.7.egg/engineio/middleware.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/python_engineio-3.4.3-py2.7.egg/engineio/exceptions.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/python_engineio-3.4.3-py2.7.egg/engineio/client.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/python_engineio-3.4.3-py2.7.egg/engineio/async_drivers/threading.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/python_engineio-3.4.3-py2.7.egg/engineio/async_drivers/gevent_uwsgi.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/python_engineio-3.4.3-py2.7.egg/engineio/async_drivers/gevent.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/python_engineio-3.4.3-py2.7.egg/engineio/async_drivers/eventlet.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/python_engineio-3.4.3-py2.7.egg/engineio/async_drivers/init.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/python_engineio-3.4.3-py2.7.egg/engineio/init.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/meld3-1.0.2-py2.7.egg/meld3/test_meld3.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/meld3-1.0.2-py2.7.egg/meld3/meld3.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/meld3-1.0.2-py2.7.egg/meld3/_compat.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/meld3-1.0.2-py2.7.egg/meld3/init.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/coverage-4.5.2-py2.7-linux-x86_64.egg/coverage/xmlreport.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/coverage-4.5.2-py2.7-linux-x86_64.egg/coverage/version.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/coverage-4.5.2-py2.7-linux-x86_64.egg/coverage/templite.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/coverage-4.5.2-py2.7-linux-x86_64.egg/coverage/summary.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/coverage-4.5.2-py2.7-linux-x86_64.egg/coverage/results.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/coverage-4.5.2-py2.7-linux-x86_64.egg/coverage/report.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/coverage-4.5.2-py2.7-linux-x86_64.egg/coverage/pytracer.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/coverage-4.5.2-py2.7-linux-x86_64.egg/coverage/python.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/coverage-4.5.2-py2.7-linux-x86_64.egg/coverage/plugin_support.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/coverage-4.5.2-py2.7-linux-x86_64.egg/coverage/plugin.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/coverage-4.5.2-py2.7-linux-x86_64.egg/coverage/pickle2json.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/coverage-4.5.2-py2.7-linux-x86_64.egg/coverage/phystokens.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/coverage-4.5.2-py2.7-linux-x86_64.egg/coverage/parser.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/coverage-4.5.2-py2.7-linux-x86_64.egg/coverage/multiproc.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/coverage-4.5.2-py2.7-linux-x86_64.egg/coverage/misc.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/coverage-4.5.2-py2.7-linux-x86_64.egg/coverage/html.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/coverage-4.5.2-py2.7-linux-x86_64.egg/coverage/fullcoverage/encodings.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/coverage-4.5.2-py2.7-linux-x86_64.egg/coverage/files.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/coverage-4.5.2-py2.7-linux-x86_64.egg/coverage/execfile.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/coverage-4.5.2-py2.7-linux-x86_64.egg/coverage/env.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/coverage-4.5.2-py2.7-linux-x86_64.egg/coverage/debug.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/coverage-4.5.2-py2.7-linux-x86_64.egg/coverage/data.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/coverage-4.5.2-py2.7-linux-x86_64.egg/coverage/control.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/coverage-4.5.2-py2.7-linux-x86_64.egg/coverage/config.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/coverage-4.5.2-py2.7-linux-x86_64.egg/coverage/collector.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/coverage-4.5.2-py2.7-linux-x86_64.egg/coverage/cmdline.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/coverage-4.5.2-py2.7-linux-x86_64.egg/coverage/bytecode.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/coverage-4.5.2-py2.7-linux-x86_64.egg/coverage/backward.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/coverage-4.5.2-py2.7-linux-x86_64.egg/coverage/backunittest.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/coverage-4.5.2-py2.7-linux-x86_64.egg/coverage/annotate.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/coverage-4.5.2-py2.7-linux-x86_64.egg/coverage/main.pyc: remove failed: No such file or directory
warning: file /opt/rockstor/eggs/coverage-4.5.2-py2.7-linux-x86_64.egg/coverage/init.pyc: remove failed: No such file or directory
Verifying : rockstor-3.9.2-49.x86_64 1/2
Verifying : rockstor-3.9.2-48.x86_64 2/2Updated:
rockstor.x86_64 0:3.9.2-49Complete!
Loaded plugins: changelog, fastestmirror Loading mirror speeds from cached hostfile
- base: mirror1.hs-esslingen.de
- epel: mirror.nextlayer.at
- extras: mirror1.hs-esslingen.de
- updates: mirror1.hs-esslingen.de
Package rockstor-3.9.2-49.x86_64 already installed and latest version Nothing to do
@upapi_rockstor Glad you got it sorted re:
And thanks for the update.
On a quick look this is actually ‘normal’ but it does appear we have some work to do here. During a Web-UI initiated upgrade, and only then, we wipe all pyc files thus:
This was added ages ago to ensure all new python code was honoured post upgrade as we had instances of older pyc’s still being in play. At least that’s what I’m assuming. But it may, such as in your case, be covering up an otherwise failed rpm update and thus having unwanted side effects. I.e. the log shows a successful verification of both the new and old rpm, where as if these failed then you would not have ended up with a broken install; maybe. I don’t think so currently but something happened and so thanks for highlighting this uncouth area of our rpm upgrade. Their may well be dragons there.
Well done persevering and I’m relieved your now sorted. Thanks again for posting your findings, I hadn’t previously though this ‘issue’ was of any consequence but maybe it is in fact masking an otherwise useful check of some sort. I’m due shortly to work in that area so I’ll have a think and experiment a little. Thanks for the heads-up.
I’d like to add a ‘me too’ for me as well. My web GUI was working prior to the eggs package update is again after another reboot. I haven’t as yet checked the logs to see if any further updates have been applied.
@Bazza Welcome to the Rockstor community.
Thanks for confirming another instance of this. Can you clarify if you needed to re-install rockstor to get this working again or did you just have to reboot ?
Thanks.
Same slowness and system info issues as OP. The actual Samba NFS performance had no issues. Giving “yum downgrade rockstor” a go.
edit: using putty not webgui to downgrade
Edit2: Ran downgrade and update 2 times no luck
@Warbucks Hello again. Thanks for chipping in on this one.
Have you tried a reboot, it may be that all that is required in these “slow and no header info” Web-UI issues is a reboot, or a:
systemctl stop rockstor
systemctl start rockstor
possibly.
Pretty sure it’s an instance of an old library still in play some how causing a mismatch, and that libraries code may even be in browser case.
Although the stop and start service is done as part of the Web-UI update. Still a bit of a mystery this one as only seems to affect a few folks.
Haven’t seen this myself as of yet though. This update was much larger than usual but is well worth persevearing as their actually, ironically, quite a few Web-UI speed-ups 
by way of long overdue optimisation of disk / pool / share management.
Hope that helps.
@Flox @upapi_rockstor @Bazza @Warbucks More info on this one:
I may have a reproducer / more info here using chrome version:
Chrome Version 74.0.3729.169 (Official Build) (64-bit)
With repeated:
“WebSocket connection to ‘’ failed: WebSocket is closed before the connection is established.” in Dev console.
Where as the same Rockstor install via:
Firefox 60.8.0esr (64-bit)
Works as intended. But I have seen some dashboard freezes but no Web-UI pauses or blank Web-UI top right.
Their is an intention to completely replace the way the dashboard works as it has tripped us up a few times in the past. But for this we need to move to Python 3 and Django 3: both non trivial tasks. All doable in time though.
Can folks having issues with 3.9.2-49 showing a mostly blank top right Web-UI and a bunch of empty dashboard widgets:
Chrome:
and some really annoying 10-30 second pauses upon visiting any page !!!
Please report your findings using Firefox.
Firefox:
If you find, having visited the Web-UI with Chrome, that the CPU load in Rockstor is consistently high their after (data-collector spinning away to itself), I have seen this once so far, then a reboot or possibly a rockstor.service stop start, as in my previous post, should calm things down again.
@Flox I’m also seeing this Chrome incompatibility on our pre-release testing channel openSUSE rpms so fairly generic problem.
My report of my installation working correctly was premature… it is working in firefox (but this has an existing browser cache. I’ll clear this shortly and retest) However in chrome which has never been used against my installation, exhibits the partial GUI problem. I also get the same issue when I test from an Android tablet.
Just to add to your findings: I’ve had this problem using
If you would like any specific information, please let me know but it may take a day or two for me to respond. My environment, if this helps.
Client laptop running Fedora 30 :-
Firefox - 69.0.1 (64 bit) - Appears to be working, except that Web GUI is much slower than before and is causing firefox generate a high cpu load on my laptop. I am seeing the following recurring errors from the Firefox dev console (changed websocket IDs)
Firefox can’t establish a connection to the server at wss://rockstor/socket.io/?EIO=3&transport=websocket&sid=################. socket.io.js:6605:34
The connection to wss://rockstor/socket.io/?EIO=3&transport=websocket&sid=################# was interrupted while the page was loading. socket.io.js:6605:34
Chrome - Version 77.0.3865.90 (Official Build) (64-bit) - I am seeing the same errors in the dev console as reported.
My Rockstor server:
HP Gen 10 Microserver
Rockstor 3.9.2.49 however /etc/redhat-release still says “Rockstor release 3.9.23 (Core)”
Linux: 4.20.12-1.el7.elrepo.x86_64 (as latest Rockstor kernel is not compatible with this hardware)
Very high CPU usage by data-collector process
Recent updates
39 | root <root> | 2019-10-05 06:25 | Update | 1
38 | root <root> | 2019-10-02 16:37 | Update | 1 EE < Rockstor update
37 | root <root> | 2019-09-27 06:50 | Update | 1
36 | root <root> | 2019-09-18 08:42 | I, U | 235 < sizeable CentOS changes
yum history info 38 - successful, except for all the eggs warning (not copied here)
Loaded plugins: changelog, fastestmirror
Transaction ID : 38
Begin time : Wed Oct 2 16:37:04 2019
Begin rpmdb : 570:7d89451c4119e4668c61b06988b6357481ae9fb5
End time : 16:37:11 2019 (7 seconds)
End rpmdb : 570:5362908611838f4d8830eb14ecda0eb8aac5c714
User : root
Return-Code : Success
Command Line : --setopt=timeout=600 -y update
Transaction performed with:
Installed rpm-4.11.3-40.el7.x86_64 @base
Installed yum-3.4.3-163.el7.centos.noarch @base
Installed yum-plugin-fastestmirror-1.1.31-52.el7.noarch @base
Packages Altered:
Updated rockstor-3.9.2-48.x86_64 @Rockstor-Stable
Update 3.9.2-49.x86_64 @Rockstor-Stable
Scriptlet output:
1 systemctl daemon-reload done
Barry
Yes firefox (android) looks fine and I is pretty snappy navigating the WebUI.
Chrome(windows or android) on the other hand is same as the OP and I had previously.
In firefox while the CPU load section does jump up to 95% initially. After a few seconds it seems to settle to ~5% with a few bumps up to 25% on one of the cores. Which with my Intel DN2800mt is pretty typical i believe.
@upapi_rockstor Thanks for the other browser tests.
I believe they are all now based on webkit so essentially the same browser base as chrome.
@Bazza Thanks for the nice system report.
Yes, no worries their. The updating of redhat-release and other os version indicators was never really maintained. We are to leave these alone in our comming openSUSE bases as it just caused incompatibilities when folks tried to compile/install stuff and unsurprisingly that stuff new nothing of “rockstor 3” etc.
Your Firefox report is currently an anomaly but is a new shiny version so that may be triggering something that looks pretty similar.
Try closing all browsers accessing your Rockstor box, reboot the rockstor box, then re-try with your version of Firefox.
@Warbucks Re:
Yes this is my experience on an openSUSE desktop system.
Yes, upon first visiting the Dashboard it does some yum / package checks and these often peak and then sustain a partial load on CPU but should then die off. 1% on moderate CPU cores is common for data-collector. But client CPU is hit pretty hard with our current dashboard.
Thank to all for your reports. I’m still not able to reproduce major slowdowns in Firefox and 3.9.2-49 should be noticeably more responsive than 3.9.2-48 and all for some time before it so any slowdown is most likely this browser python-enginio / gevent incompatibility or possibly quotas being re-enabled by a docker-ce install or the like. Normally only relevant on a re-install though and when then moving to our newer docker-ce from the prior version.
Linking for context to a prior pull request that ended up not being required as python-engineio ended up upgrading again and what problems we saw back then disappeared again. Looks like we may be hitting something similar again. This situation is complicated by our still using Python 2.7 so we do have some technical debt related issues here:
https://github.com/rockstor/rockstor-core/pull/1996
I’ll play with some of these library versions again and if a library pin or 2 works then I can roll out another stable rpm to hopefully get this sorted again.
If anyone else gets around to building from source to play with these library versions then do post here. I’m currently working on Web-UI upgrade within our openSUSE variants to help with moving in that direction but should be finished pretty soon. Their after I’ll have a look myself and if any joy will roll and release another stable rpm. My bet currently is on the same library pinning as was indicated in that pr of mine referenced above. So that’s probably where I’ll start when I’m done with my current task.
Apologies to all for the inconvenience, and the current work around looks to be:
“Use Firefox”,
at least mostly. But yes, almost all other browsers are webkit so we need to get this sorted.
Post here if you end up playing with source builds and newer / older library versions. But note that this should not be done on any production systems as a fresh source build will wipe the central Rockstor db.