Fresh install - upgrade breaks

1

So i did a fresh install , and upgraded everything, my setup broke. and I broke it more. so now i am back to a fresh install and up and running .

i would like to know why it broke when upgrading to kernel 4.12 from 4.10

can someone inspect my hardware setup for anything that might stand out? to not supporting 4.12?

4.12 would not boot after clicking update in rockstor.

Linux version 4.10.6-1.el7.elrepo.x86_64 (mockbuild@Build64R7) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-11) (GCC) ) #1 SMP Sun Mar 26 12:19:32 EDT 2017
[notrin@def

Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 2
On-line CPU(s) list: 0,1
Thread(s) per core: 1
Core(s) per socket: 2
Socket(s): 1
NUMA node(s): 1
Vendor ID: AuthenticAMD
CPU family: 16
Model: 6
Model name: AMD Athlonā„¢ II X2 240 Processor
Stepping: 2
CPU MHz: 800.000
BogoMIPS: 5600.34
Virtualization: AMD-V
L1d cache: 64K
L1i cache: 64K
L2 cache: 1024K
NUMA node0 CPU(s): 0,1

00:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD] RS880 Host Bridge
00:01.0 PCI bridge: Dell Device 9602
00:07.0 PCI bridge: Advanced Micro Devices, Inc. [AMD] RS780/RS880 PCI to PCI bridge (PCIE port 3)
00:11.0 SATA controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 SATA Controller [AHCI mode]
00:12.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
00:12.1 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0 USB OHCI1 Controller
00:12.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
00:13.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
00:13.1 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0 USB OHCI1 Controller
00:13.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 USB EHCI Controller
00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 SMBus Controller (rev 3c)
00:14.2 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 Azalia (Intel HDA)
00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD/ATI] SB7x0/SB8x0/SB9x0 LPC host controller
00:14.4 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 PCI to PCI Bridge
00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 10h Processor HyperTransport Configuration
00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 10h Processor Address Map
00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 10h Processor DRAM Controller
00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 10h Processor Miscellaneous Control
00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 10h Processor Link Control
01:05.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] RS880 [Radeon HD 4200]
01:05.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] RS880 HDMI Audio [Radeon HD 4200 Series]
02:00.0 Ethernet controller: Broadcom Limited NetLink BCM57788 Gigabit Ethernet PCIe (rev 01)

Bus 002 Device 002: ID 058f:6362 Alcor Micro Corp. Flash Card Reader/Writer
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 006 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 005 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 002: ID 413c:2105 Dell Computer Corp. Model L100 Keyboard
Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 003 Device 002: ID 046d:c016 Logitech, Inc. Optical Wheel Mouse
Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub

[notrin@defragged ~]$ free -m
total used free shared buff/cache available
Mem: 2755 299 78 16 2377 2222
Swap: 2816 16 2800

2

@notrin Welcome to the Rockstor community and thanks for bringing your prior email support query to the forum. There are more eyes here and certainly more expertise. Iā€™ll do what I can to chip in here so we might sort what issues you may have run into.

We have had some reports of the 4.12 kernel not working where there was prior function with 4.10. But to flesh out the particular upgrade problems you encountered a little more I understand, from the email thread that lead to this forum post, that you also had issues with samba not working after an upgrade. This is most likely unrelated to the kernel failure and there is a little history of samba failing after a certain update, especially where that update was interrupted by a premature reboot. Iā€™m not sure of the history in your case but just trying to tease out what we can look at independently here in the hope that we can divide and conquer.

So to summarise you updated to stable release, presumably from a 3.9.1-0 iso install. And given we have not release an iso for a very long time that ends up being quite a large update which incorporates an upstream CentOS version shift. Partly our failing due to long periods between isos but there are plans and mechanisms in the works to address this: but for now we have to go from where we find ourselves.

So given you have a stable subscription Iā€™m going to suggest that we try and divide out your kernel issue from the rest of your system, given that a successful update should still allow you to boot into the older kernel. So lets get that bit sorted first.

Could you confirm the version of Rockstor you are currently running? I.e. top right of Web-UI and due to a bug when moving from testing to stable to also confirm an installed stable version via:

yum info rockstor

So assuming you are currently running 3.9.1-0, ie a non updated iso install. There is some sense in first updating to the latest testing channel release; as the resulting 3.9.1-16 was also released as the first of the next run of 3.9.2 stable channel versions. We droped testing channel releases after that point due to other, mostly resource related, background issues; but there is a will to re-establish this channel in time. Please see the following forum thread ā€˜Introā€™ / ā€˜Quick historyā€™ sections for some context:

So if you are on at least 3.9.1-0 I suggest that you first try updating to the latest testing channel version, by way of a step towards getting you ultimately to the latest stable, but via the command line so we can see what might be ā€˜breakingā€™ if in deed it does this time. So if on at least 3.9.1-0, ie last released clean iso install, subscribe to the testing channel first via the Web-UI but donā€™t initiate the update itself via the Web-UI. I suggest you then initiate the update on the command line there after thus:

yum update

That way we can see how it goes.

This should also bring with it the 4.12 kernel but if the update is successful this time we can work our way back around to the 4.12 kernel issue. For now once the update is complete and assuming it gives no errors you can reboot (but only once itā€™s actually finished) and re-select the 4.10 kernel to try and keep our changes to a minimum for now.

Lets take it from there and see how that goes. That update, if from 3.9.1-0 is still going to be very large as the rest of the system will still be updated to the most recent upstream CentOS. But at least this way we can see where / if the update fails this time.

Hope that helps and do past the ā€˜yum updateā€™ output in the terminal. What we are looking for is a potential repeat of such events as previously broke the samba update that was associated with changes at around CentOS 7.4 ie see:

for some reports of broken updates breaking samba. @Haioken, @Bert, and @doenietzomoeilijk worked on this problem in that thread and I saw it myself temporarily on systems here:

But an interrupted update could have frozen this ā€˜broken sambaā€™ state in-place and a whole host of other potential issues.

See how you go and if in doubt just post the output of:

yum info rockstor

and we can take it from there.

1 Like
3

wow, awesome response time! I really appreciate that. Thanks much. I will look into this probably this weekend. Thanks.

4

Hello

Finally got to do this.

I had a internal network issue i needed to resolve first,

[root@defragged ~]# yum info rockstor
Loaded plugins: changelog, fastestmirror
Loading mirror speeds from cached hostfile

Installed Packages
Name : rockstor
Arch : x86_64
Version : 3.9.1
Release : 0
Size : 84 M
Repo : installed
From repo : anaconda
Summary : RockStor ā€“ Store Smartly
License : GPL
Description : RockStor ā€“ Store Smartly

[root@defragged ~]# yum update
No packages marked for update

what next? just boot to the new kernel?

5

@notrin Hello again.

By new kernel you are presumably referring to the 4.12 kernel but that comes from either the testing or stable updates repos and from your ā€˜yum info rockstorā€™ command showing no available rockstor versions, and only the one included with the iso, you have yet to subscribe to either repository.

So yes you could now try and see if all is as expected after a reboot now that all the upstream (CentOS) updates are in place, and presumably without error given the output of your ā€˜yum updateā€™ command, and you presumably noticed there were a tone of them.

This way we can at least narrow down which updates are throwing your systemā€™s function off.

So you might as well reboot from where you are as Iā€™m unsure if you have yet done so after having installed at least the upstream updates. And given you are yet to have the potentially problematic 4.12 kernel on your system, Iā€™m anticipating a successful boot and Rockstor Web-UI login there after. Assuming this is the case and you have already installed, via the ā€˜yum updateā€™ command all bar the Rockstor specific updates you might as well go straight for the stable subscription, give you already have one, plus itā€™s way newer code. There may there after be a db migration issue (sorted in a more recent release) so you might there after only get full function after another reboot. And if you are still unsure of the 4.12 kernel we can keep that issue separate for the time being by you selecting the 4.10 again on reboot, even though after selecting an update channel you should then receive the 4.12 as an update. That way at least we can see if all functions prior to finally doing yet another reboot and finally selecting the 4.12. All in an effort to narrow down your issue and at least get you up and running with latest code, if not latest Rockstor provided kernel.

So short of it is:
Try reboot from where you are.
if boot / Web-UI OK, then subscribe to stable (using your existing credentials).

  • assuming this goes OK then do another:
yum update

to pull in the new kernel, and the rockstor associated docker-ce (if stable subscription) and this way we can see if any errors occur at this point.
if that all looks to have gone in ok and finished, give the Rockstor Web-UI time to re-set itself as there are quite a few database migrations that have to take place. Once that done and the Web-UI re-appears try another reboot and select 4.10.

If you get to new rockstor version Web-UI after the reboot with 4.10 give it a moment and then try a reboot and select 4.12 to see if that works this time. If not then you are stuck for the time being on 4.10 as we can take it from there.

Lots in one go but give we have quite a delay between us I though it best to sketch out a little further in the hope that we can get you further along with each forum cycle.

Hope that helps and remember to past the output of all command you run and the reboots you made into this forum thread to make it possible for folks here to track what has happened. Iā€™m assuming your first yum update command succeeded in installing successfully all of the hundreds of upstream CentOS packages, you didnā€™t post the output or mention this. If not then make sure your install has an internet connection and re-do the ā€˜yum updateā€™; then follow the above guide. We really need a newer iso but Iā€™m working on that for our openSUSE release.

6

Hello, sorry, i did the yum update first time on the machine, so i could not copy paste, i meant to paste that yum update to imply that it was ran and everything there was up to date, my bad.
thanks so much for being so detailed with support i appreciate it greatly. I will do better to respond in a more timely fashion.

i did yum update and no packages are marked for update.

[root@defragged ~]# yum update
Loaded plugins: changelog, fastestmirror
Loading mirror speeds from cached hostfile

  • updates: mirrors.ocf.berkeley.edu
    rockstor | 2.9 kB 00:00
    No packages marked for update

Installed Packages
Name : rockstor
Arch : x86_64
Version : 3.9.1
Release : 0
Size : 84 M
Repo : installed
From repo : anaconda
Summary : RockStor ā€“ Store Smartly
License : GPL
Description : RockStor ā€“ Store Smartly
i have booted to the 4.12 successfully and i see this error when i login to the dashboard.
i assume at this point i should enable udpates in rockstor?

You are running an unsupported kernel(4.12.4-1.el7.elrepo.x86_64). Some features may not work properly. Please reboot and the system will automatically boot using the supported kernel(4.10.6-1.el7.elrepo.x86_64)

i notice that there is some new folders in myshare now
here is the output of ls of the shared folder.

btrfs network trust
containers plugins volumes
image swarm
tmp

i look forward to the opensuse release!

7

@notrin Sorry for my slow response. Iā€™ve been fairly distracted working on a web app to allow stable subscribers to manage their Appliance IDs.

Yes me too. It will be my main priority once the backend changes are more settled and those changes actually include openSUSE specific rpm repos so there is some headway being made on this front.

Now to your problem. So at least you have received and booted into the 4.12 successfully. The Rocksor Web-UI error is just a warning that is intended to indicate a need to reboot to pickup a newer kernel after an update has happened. Only in your case the mismatch is backwards as you have the newer kernel and an older rockstor which in turn is checking for a 4.10 kernel and as you say finds a 4.12.

Itā€™s actually a little confusing as you are offered no rockstor package updates yet you have received the 4.12 kernel. Thatā€™s rather strange as the 4.12 kernel comes from either of the testing or stable channels, and from either your should be offered updates to the rockstor package also. These should show up as available from the ā€˜yum info rockstorā€™. Could you paste the output from the following commands run as root:

yum repolist enabled

and

yum repolist disabled

Also check that you can successfully access at least the testing repo:

http://updates.rockstor.com:8999/rockstor-testing/

from your network, and your should see a raw html output on your Rockstor terminal of that same page via the following command:

curl http://updates.rockstor.com:8999/rockstor-testing/

ie a load of lines lie the following:

...
<li><a href="rockstor-3.9.1-15.x86_64.rpm"> rockstor-3.9.1-15.x86_64.rpm</a></li>
<li><a href="rockstor-3.9.1-16.x86_64.rpm"> rockstor-3.9.1-16.x86_64.rpm</a></li>
<li><a href="rockstor-3.9.1-2.x86_64.rpm"> rockstor-3.9.1-2.x86_64.rpm</a></li>
...

And given Iā€™m a little stuck on this one and we have only a high latency chat to get it sorted Iā€™m going to suggest that you try subscribing to the testing channel and see if that offers you anything, if so you might as well update to that while we work out why your system is not offering any rockstor packages yet you have received and installed the kernel update from the same repositories. Then we can take it from there.

Sorry to not be of much use on this iteration. Itā€™s probably something silly that Iā€™m missing.

Let us know if you are offered a rockstor package update once you subscribe to the testing channel and we can take it from there. And remember that the canonical info of what is installed and available is given from:

yum info rockstor

and to update only the rockstor package, if one is found, the following should work:

yum update rockstor

If testing offers you an update pop it in via that last command to make sure we catch any potential errors. And if that works, and after an update, try moving back to stable subscription and again check via those 2 commands.

Apologies for not having twigged the cause of this just yet.

Anyone else see why @notrin should have received all updates, including kernel, bar the rockstor package itself?

8

Thanks, I find your responses to be of a very appropriate response time and very detailed, I thank you. I will get to this as soon as I can.

9 
[root@defragged ~]# yum repolist enabled
Loaded plugins: changelog, fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirror.hostduplex.com
 * epel: mirror.prgmr.com
 * extras: mirrors.xmission.com
 * updates: mirror.facebook.net
repo id             repo name                                           status
base/x86_64         CentOS-7 - Base                                       10,019
epel/x86_64         Extra Packages for Enterprise Linux 7 - x86_64        13,038
extras/x86_64       CentOS-7 - Extras                                        385
!rockstor           Rockstor 3 - x86_64                                       55
updates/x86_64      CentOS-7 - Updates                                  1,463+30
repolist: 24,960

[root@defragged ~]# yum repolist disabled
Loaded plugins: changelog, fastestmirror
repo id                       repo name                                         
base-debuginfo/x86_64         CentOS-7 - Debuginfo                              
base-source/$releasever       CentOS-$releasever - Base Sources                 
centosplus/x86_64             CentOS-7 - Plus                                   
centosplus-source/$releasever CentOS-$releasever - Plus Sources                 
epel-debuginfo/x86_64         Extra Packages for Enterprise Linux 7 - x86_64 - D
epel-source/x86_64            Extra Packages for Enterprise Linux 7 - x86_64 - S
epel-testing/x86_64           Extra Packages for Enterprise Linux 7 - Testing - 
epel-testing-debuginfo/x86_64 Extra Packages for Enterprise Linux 7 - Testing - 
epel-testing-source/x86_64    Extra Packages for Enterprise Linux 7 - Testing - 
extras-source/$releasever     CentOS-$releasever - Extras Sources               
fasttrack/$releasever/x86_64  CentOS-7 - fasttrack                              
updates-source/$releasever    CentOS-$releasever - Updates Sources              
repolist: 0

sweet. looks good after enabling auto updates

[root@defragged ~]# yum info rockstor
Loaded plugins: changelog, fastestmirror
Loading mirror speeds from cached hostfile

Installed Packages
Name : rockstor
Arch : x86_64
Version : 3.9.2
Release : 48
Size : 85 M
Repo : installed
From repo : Rockstor-Stable
Summary : RockStor ā€“ Store Smartly
License : GPL
Description : RockStor ā€“ Store Smartly

[root@defragged ~]# yum update rockstor
Loaded plugins: changelog, fastestmirror
Loading mirror speeds from cached hostfile

rockstor | 2.9 kB 00:00
No packages marked for update

I thank you for your support; I will be recommending your project to all of my tech friends.

Thanks so much!.

10

@notrin Glad your now sorted and thanks for the command outputs and update.

OK, curios. Thanks for sharing your findings. Iā€™ll look into this when next in that area, we may have a bug in how we manage these repositories that the auto update setting allows another go at setting. From your initial ā€˜yum repolist enabledā€™ report you had no Stable channel repo enabled ie the following line didnā€™t show:

!Rockstor-Stable                           Subscription channel for stable updates                                         60

Where as given you now have yesterdays release of 3.9.2-48 Iā€™m guessing that has now been sorted.

So thanks, it looks like youā€™ve uncovered a bug and shared a work around at the same time. Nice. Iā€™ve made a GitHub issue to keep an eye on this one:

Thanks again for helping to support Rockstorā€™s development via a stable channel subscription and engaging in this some what elusive bug tracking exercise. And apologies for the rough ride, I just havenā€™t see this happen before.

1 Like