Built on openSUSE testing channel live (early-adopters/developers only)

After the update the shares on my root pool seem a bit messed up. The pool previously had snapshots enabled (default setting, I never did anything to the root pool). All the custom shares (so the non-home share) are now at /@/.snapshots/1/snapshot/ and seem to have lost all data.

Further, the root pool now has a red “quotas disabled” status.

EDIT: I just checked, the data is still there on disk, it’s just that the path of the shares no longer matches.

@freaktechnik Hello again.
Did you read the bit about having to reboot.

Have you now rebooted and still see issues on the root pool re default shares.

Hope that helps.

Yes, I have rebooted twice. The home share (/@/home) was migrated correctly, but any other shares on the root pool have not had their mount point adjusted.

@freaktechnik Could you start a new thread on this.

But note that user shares on the root pool were non functional for boot-to-snap config in openSUSE, hence the fix. So any you made were likely in the wrong place and if created from Rockstor’s Web-UI, and boot-to-snap is configured as you indicate with the .

bit then they would have disapeared upon page reload.

Only shares created after 3.9.2-58 via the Web-UI on ROOT in the ‘Built on openSUSE’ will be in the correct place. Prior created ones are going to have to be deleted as they existing in the wrong place and will likely be lost to the Web-UI on a boot-to-snap rollback. You will likely have to remove all prior user created shares to remove the confusion.

Lets continue in another thread as your shares are likely non standard unless they were created within 3.9.2-58. Also log counterparts to indicate exactly what’s happened and a subvol list via command line in that thread will help. But likely you just need to clean up (delte) those ‘confused’ shares as we were essentially doing it wrong before 3.9.2-58 on the boot-to-snap config and I believe we are now doing it correctly. Plus of all the default shares only “home” should be surfaced. No other server install defaults should show up at all in the Web-UI.

Hope that helps.

1 Like

will there be migration process from centos to opensuse? Id be willing to schedule a cut over if I can import my shares and config.

Running mine on a

  • ProLiant DL380 G7
  • Processor Type:Intel® Xeon® CPU X5690 @ 3.47GHz
  • Logical Processors:24
  • Ram 143.99 GB
  • Disk : 2x 1.8TB ent Micron ssd. ( pool ) 2x 1tb pioneer ssd. 3x 500gb ssd. 1x 200gb ssd.
  • Networking 10GB

@Roger_Lund Hello again,

That’s a nice machine :slight_smile: .

In as far as config back and restore post re-install, yes. And if you use a different system disk then if all falls on it’s face you can simply boot from your old system drive and it’s shouldn’t have noticed anything having happened. But probably best to try and avoid going backwards in kernel version. Just in case. Btrfs on disk format has not changed though but still, there have been now hundreds of btrfs fixes between our now legacy CentOS base and our ‘in testing’ Leap15.2 RC base.

Configuration Backup and Restore

So once the new system is installed one would import the pool as normal and then apply the previously saved and downloaded config backup, then apply it. Not perfect as doesn’t cover everything but the pool import is the same code we have been developing all along and it’s now fairly stable. Or at least we haven’t had a report of it failing for some time now.

For more up-to-date info on the ‘Built on openSUSE’ variant I’ve now transitioned to posts it’s progress on the new ‘Beta’ thread:

In your particular hardware case though the following issue may be relevant, but probably only if you have a similar disk controller. But I’m pretty sure I can fix that issue once I find a moment to concentrate on it:

https://github.com/rockstor/rockstor-core/issues/2126

Check out that issue and see if it is likely that you have the same controller as if so you will want to wait until that is fixed on our side.

Hope that helps, but as yet we haven’t promoted any of the testing rpms to the ‘built on openSUSE’ Stable channel as they are just not quite there yet.

1 Like

In this case, i’m running two SAS3008 PCI-Express Fusion-MPT SAS-3 cards.

I think i’ll have to grab another USB drive to install to next time i’m in the pop, and attach it.

1 Like

Hi All

Since it seems that Rockstor 4 is a stable release candidate, I took the plunge this weekend to build up my OpenSUSE system on a new SATA SSD (no native NVMe booting on my old motherboard). The old one is in a safe place for now (looking like I won’t need it though - only missed a single VM disk image that was on a share in the CentOS root partition, since copied over to the same place as the rest of them)

Pools were imported fine (despite starting to get curtailed API responses from the “Shares” page on my old CentOS based system, causing the page to not render and no rockons could be added due the same API web call curtailing). Also I moved my appliance ID over just fine thanks to the tools for it (it took me longer to find my activation key from 2 years ago).

Only thing that didn’t seem to work was the importing of my Samba or SFTP share list - which is no great loss, as I did not have that many and I read the config backup by eye to recreate them. Interestingly NFS worked perfectly fine. Maybe I was just impatient, if I get a chance I might build a VM at some point with a spoofed share list in a pool and see if it happens again. I’ve rebooted a couple of times since (network config changes - so might not have the original logs around import now, but will see)

Also, if anyone has a SuperMicro X10-SLH/X10-SLM, for the LEAP 15.2 install, UEFI booting is OK, but might want to disable secure boot on the summary before package installs. For me the installer would not complete (no activity shown after packages completion) with secureboot enabled - could just a red herring though, I use a pile of repurposed ex-workstation bits and pieces.

System seems nice and quick now, loving it. OpenSUSE package for KVM is great too, no more git pulls for the UEFI support and dodgy XML hacks to use it - it’s baked in.

2 Likes

I am not sure if this is the correct thread to post issues, however after the install (OpenSUSES 15.2, installer iso created with kiwi-ng on freshly installed and updated 15.2) i keep getting those messages on the console and in dmesg:

[404887.813445] audit: backlog limit exceeded
[404887.813505] audit: audit_backlog=65 > audit_backlog_limit=64
[404887.815997] audit: type=1400 audit(1599054448.661:817848324): apparmor="DENIED" operation="ptrace" profile="docker-default" pid=22193 comm="apps.plugin" requested_mask="read" denied_mask
="read" peer="unconfined"
[404887.817281] audit: audit_lost=12218207 audit_rate_limit=0 audit_backlog_limit=64

The rate is one event every second.
Upon checking AppArmor is disabled and auditd is not present on the system.
Could you please advise?

@lexxa First off well done on a successful installer build.

As to your log entry, I’ve not yet seen this myself. Is this directly from the resulting install. Or did it appear after applying updates, or have you, since install, installed anything. And if so what.

Let us know if you end up tracking this one down.

If anyone else is seeing this, or is more up on what this is about then please chip in. I’m afraid I’m a little tied for time at the moment so can’t really look into this apparent log spam right now.

Hope that helps.

Hi @lexxa,

Tahnsk for the report… Similar to @phillxnet, I’ve not seen that one, but it may be container-specific… I presume you have a rock-on (or at least a docker container) running, isn’t it? If yes, could you turn it/them off and identify which one(s) is causing this log event?

I can’t have a look at my systems at the moment but maybe the docker documentation on their AppArmor profile may help:

AppArmor shouldn’t be running indeed, but maybe we have not completely disabled it or something else re-triggered it… you did specify auditd wasn’t running, though… curious.

We’ll hopefully get to the bottom of this and identify whether we should tweak our docker settings accordingly.

1 Like

@lexxa,

Following-up on which rock-on you have currently running… would you have the Netdata rock-on running, by any chance? If yes, I might have an idea.

1 Like

@lexxa,

Sorry for the message spam, but I was convinced it was indeed Netdata and dug up a little more.
It turns out it actually is Netdata generating this error, and it is simply a sign we need to update our rock-on accordingly.

Briefly, there is now an official docker image for Netdata so we need to switch to that one and update our rock-on config accordingly. In particular, we need to add a dedicated apparmor setting to our docker run command for this rock-on: --security-opt apparmor=unconfined.

This should fix it.

@lexxa, as you noticed this error, would you fancy creating a corresponding issue on Github linking to your message here? You would have credit, this way.

1 Like

@phillxnet Hello, the task flow resulting the described behavior was ISO build (quite easy actually, and this comes from a Windows Engineer!), install, zypper refresh, zypper up --no-recommends, reboot, pool import, definition of rockon storage and install of Netdata and Duplicati rockons

@Flox yes indeed, as mentioned above one of the two rockons installed is netdata (went into the shell and enforced execution of the bash version of the lm-sensors, other than that unaltered docker image)

As per github issue, will try to create one in as much detail as possible.

2 Likes

Thanks a lot for creating this issue, @lexxa! I’m linking to it below for reference:

Also, note I submitted a corresponding pull request last night:

I won’t be able to merge it immediately as there are a few backend things that need to be worked out (unrelated to this PR specifically), but we should be able to have that merged and published once it’s compete.

Thanks again for your report, this is an important update that needed to be done so I’m grateful you noticed it and brought it up!

1 Like

I am coming with yet another issue.
On the latest build of rockstor with fully update SUSE.
I am running the duplicati rockon.
Once a scheduled backup starts RAM usage remains basically the same but all available ram gets “cached”. After the job finishes the RAM is still cached and sooner or later the OS gets to write to /swap. After a second run of the backup job /swap gets filled considerably.
Once the backup job finishes the only way to “un-cache” the system is a reboot.

I am ok with using swap but would like another processes (nfs/cifs) to use the availabale ram. I am not sure how to drop the cached pages (sync; echo 3 > /proc/sys/vm/drop_caches) gets the job done but I would hate to introduce inconsistency to the memory content doing this too often.

Any advice how to further troubleshoot or report the issue would be greatly appreciated

Hello again, @lexxa

That unfortunately seems like an issue with Duplicati rather than Rockstor. I quickly lookep it up and it seems to have been a “somewhat common” issue with Duplicati. For instance, I found an issue on their Github page with similar behavior suspecting a memroy leak that seems to have been fixed by a Duplicati update:

I would refer you to the Duplicati project for help about that, unfortunately, as they would be the best positioned to help. The only thing I would suggest is to make sure you are running the latest docker image.

I’m sorry I couldn’t be of more help here.

2 Likes

Bit of an update for whoever might be interested really.

Over the past couple of months I’ve had x86_64 Rocksor 4 running on my home NAS without any issue. Well apart from a motherboard failure, but I can’t point the finger at Rockstor for that one. It is Rock steady (pun intended), reliable and with good performance. No complaints from me or the users (ie my family) for document storage and supplying audio/video media to my LMS/Jellyfin/Icecast server.

I’ve also continued with a fair bit of successful testing on a Pi4 and some further RAID testing in x86 VMs, seeing how Rockstor handles changes to RAID levels, drive loss etc. I believe the newer kernel and BTRFS on Leap is more stable when performing constant tests like this, and more so than the same testing done on the CentOS base which I’ve done in tandem.

I’m not an AD user, so the existing issues with that (which I believe are close to being resolved anyway) don’t stop me from happily using Rockstor 4.

I think my testing has come to a close now, and as I type I’m running one last set of tests changing RAID level from 10 to 1 and other odds and sods in my VMs. As far as I’m concerned it’s fit and forget for me :slight_smile:

One last thing - I’m not a Rockon user (yet), so cannot comment on that area.

Looking forward to showtime :slight_smile:

Geoff

4 Likes

Hello,
New user (old linux hand) - successfully built system with a few 3TB drives I had lying around on an old intel i3-540.
Everything looks good - defined an NFS share and a Samba Share
I keep getting a pop-up error window which totally prevents doing anything in the GUI.
image
Would like to know how to fix this…
Going to datatables.net - recommends fixing code…

@dvgeek Thanks for the report and glad you got the new DIY installer built and deployed.

Yes this has popped up from time to time in a variety of places. The exact cause is likely a bit of us ‘doing it wrong’ somewhere but we’ve yet to narrow it down or get a reliable reproducer in order to know if we’ve fixed it.

The work around, once it occurs, is to simply use the refresh button in your browser. There after it appears to go away. Not specific to the new build but just some lingering issue we’ve had for a while. Again it’s awaiting a reliable reproducer before we can create an issue where we can test/prove whatever fix is submitted.

Hope that helps.

1 Like