Setting up Rockstor virtualized on Xen

Troubleshooting
1

Hi all,

I’m having some trouble creating a rockstor VM with two passthrough drives on a Xen XCP-ng Server (8.2).

To resume what was done, the drives were put in Passthrough using the following method:

mkdir /srv/SR_NAME
xe sr-create host-uuid=uuid_of_host name-label=”DISKS SR” name-description=”DISKS SR” type=udev content-type=disk device-config:location=/srv/SR_NAME
ln -s /dev/sdb /srv/SR_NAME/sdb
ln -s /dev/sdc /srv/SR_NAME/sdc
xe sr-scan uuid=YOUR_NEW_UUID

Created a VM with 2CPU 4GB RAM and 1st disk 16GB and attached the previous disks to the VM.
After booting the VM hit tab erase everything to dhcp to avoid the sda error, installation starts.

I select the disk for install, used manual partitioning, no /home partition, 4GB /swap, 500MB /boot and the rest of free space to /. Filesystem btrfs.

Start the install/finish the install. Reboot.
Machine boots fine. I go to the server address on the browser open the initial rockstor page put the hostname, username, password twice.

I get this error when hit continue:

##### Houston, we’ve had a problem.

Error running a command. cmd = /sbin/btrfs filesystem show /dev/disk/by-id/xvda3. rc = 1. stdout = [’’]. stderr = [‘ERROR: not a valid btrfs filesystem: /dev/disk/by-id/xvda3’, ‘’]

            Traceback (most recent call last):
  File "/opt/rockstor/src/rockstor/rest_framework_custom/generic_view.py", line 41, in _handle_exception
    yield
  File "/opt/rockstor/src/rockstor/storageadmin/views/disk.py", line 377, in post
    return self._update_disk_state()
  File "/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/utils/decorators.py", line 145, in inner
    return func(*args, **kwargs)
  File "/opt/rockstor/src/rockstor/storageadmin/views/disk.py", line 322, in _update_disk_state
    p.uuid = btrfs_uuid(dob.name)
  File "/opt/rockstor/src/rockstor/fs/btrfs.py", line 1055, in btrfs_uuid
    [BTRFS, 'filesystem', 'show', '/dev/disk/by-id/%s' % disk])
  File "/opt/rockstor/src/rockstor/system/osi.py", line 115, in run_command
    raise CommandException(cmd, out, err, rc)
CommandException: Error running a command. cmd = /sbin/btrfs filesystem show /dev/disk/by-id/xvda3. rc = 1. stdout = ['']. stderr = ['ERROR: not a valid btrfs filesystem: /dev/disk/by-id/xvda3', '']

But if I go on cli and check which fs it is:

# df -Th | grep "^/dev"
/dev/xvda3     btrfs      12G  1.6G   11G  14% /
/dev/xvda1     ext4      477M  113M  335M  26% /boot
/dev/xvda3     btrfs      12G  1.6G   11G  14% /mnt2/rockstor_rockfs3

If I reload the page I am logged in, but going to Storage > Disks is empty, and “Rescan” gives the same error as before.

2

In the meanwhile I forgot to mention I had ran yum-update and updated the system.

Ok so I noticed the path /dev/disk/by-id/ only had the qemu_dvd_drive so I created symlinks to /dev/xvda3 and hit “Rescan” no more of that error.

However I now have this:

53
533022×1974 1010 KB 

# btrfs fi sh
Label: 'rockstor_rockfs3'  uuid: 17efcfb9-d1cf-4c91-835c-cb9df3203bf8
	Total devices 1 FS bytes used 2.41GiB
	devid    1 size 11.79GiB used 3.27GiB path /dev/xvda3

Label: none  uuid: 47129961-b287-4d18-97d1-aa0a8da61731
	Total devices 2 FS bytes used 112.00KiB
	devid    1 size 1.82TiB used 2.01GiB path /dev/xvdb
	devid    2 size 1.82TiB used 2.01GiB path /dev/xvdc
3

@maverick Hello again,

The problem you are facing is that Rockstor absolutely depends upon device serial numbers, as does udev. That is why you were seeing the ‘missing’ by-id names. Simply creating these is not enough as we also resource udev to retrieve these, normally hardware supplied, unique drive identifiers. See the following technical wiki entry, Subtitle: Rockstor’s Serial Obsession, for the ‘why’:

If you can arrange for your hypervisor to create or pass-through these normally hardwre supplied drive elements then udev will be able to auto-create the required by-id names and also be able to supply these same serials to Rockstor when requested.

The updated versions of the error messages you are seeing, and have posted above, now contain links to our “Minimum system requirements” doc section here:

http://rockstor.com/docs/quickstart.html#

That docs section contains an example required configuration for VMware but not all hypervisors are equal in this capability. KVM/libvirt is also capable of supplying device serials but I’m unsure of this capability within Xen unfortunately. Hopefully other forum members can chip in with their knowledge/experience re Xen in this respect.

Also note that 3.9.1-0 is now very old. But I’m aware that we don’t yet have a newer installer; sorry about that. But you can now build your own fully updated Rockstor 4 installer and we will have downloads available as soon as this is out of release candidate phase. See the following GitHub repo for how to build our next generation installer which is now ‘Built on openSUSE’ Leap 15.2:

This new installer does not however change the serial requirement, but it would include all pending upstream updates as of it’s build date which is always nice to have.

Another point of interest here is that we may still have a bug that is affecting you. If you first resolve the serial presentation issue so that the guest auto generates the by-id names, and can produce the required matching serials on request we can then see if we have a name parsing issue with these xvda names as they look odd to me so there may still be some work for us to do there.

Hope that helps and let us know how you get on with the initial drive serial problem as it would be good to know of a robust solution in Xen that does not reside in the guest (as a work around) but in the host/hypervisor such that it presents dependable and drive-tracking capable serials.

2 Likes
4

Hi @phillxnet thank you for your amazing response.
I have been thoroughly looking around for a solution to this issue and unfortunately I’m not sure if anyone was able to overcome this issue.
We have another Rockstor install where the whole machine is dedicated to Rockstor and that’s our preferred environment for it, here I was just really trying to give a better use to this couple of disks that were attached to the hypervisor but we’re not using them.

I have to say that if there’s a guy that knows the Xen hypervisor inside out and whom I believe could be a really help on sorting those serial drive issues is Olivier Lambert from the Vates team (they develop the xcp-ng hypervisor) they are extremely knowledged and really nice people and I believe you could help each other out better than with my involvement.

I had read about the “Built on OpenSUSE” soon to come out and looking forward to it, but the disk serial issue is still a show stopper on this thing I was trying to achieve here.

Edit: P.S. if I may leave a request for the new version, please have it not disabling firewalld!

2 Likes
5

Hi @maverick. I just stumbled across this inquiry. Although my response may be significantly aged, I thought I’d respond as we have been operating Rockstor as a VM in XCP-NG for several years. We too, ran into the serial number issue and never found a reliable way to inject serial numbers on devices provided by the host. We had found mentions of a XEN related toolkit that contained an application to accomplish this task, but never successfully discovered a downloadable copy.

Our solution ( Details are in the XenServer 7 Backstory.) was PCIe passthrough of the storage controllers. This presents the actual hardware to Rockstor so all is good with the exception of a BIOS boot partition being required to properly start the Rockstor CENTOS host. We eventually moved our SWAP partition to the /dev/xvda provided disk as well to keep the Rockstor Host OS clean as we always implemented a mdraid1 mirror on SSDs.

Today, we are moving to the OpenSUSE deployment driven by the newer hardware we are utilizing for our XCP-NG deployments, i.e. AMD Ryzen 5900 series CPU and Broadcom 9500 series Tri-mode storage controllers with LSI SAS 3616 controllers that even XCP-NG 8.2’s kernel could not recognize properly. Based on successful testing of an OpenSUSE Leap 15.2 VM utilizing the aforementioned controller with 14TB SAS 12 Gbps drives, we investigated and have successfully deployed Rockstor 4.0.4-0 as a VM hosting the Rockstor OS on a XEN /dev/xvda virtual drive. Rockstor seems quite content allowing the installation of the OS in this manner. We’re excited about this approach as it allows the use of XEN snapshots, which was sorely missed in the strictly controller passthrough environment. Obviously, with passthrough controllers, Rockstor 4 works as effectively with those as Rockstor 3 achieving bare metal performance.

One last issue we did discover which does allow the utilization of XEN hosted storage repositories of passed through “disks”; if an mdraid is created with those disks utilizing the any configuration of a mdraid similar to the mirror procedure mentioned previously, Rockstor will recognize it as a “Disk Pool” that shares can be deployed upon. The key to this approach is being able to mkfs.btrfs -f -L data /dev/md### the drives while offline. We accomplished this by booting into Rockstor’s “Rescue Mode” just as building the OS on an mdraid. We did find it to be more reliable to do this after the OS has already been successfully deployed on its mdraid first. It seems if one attempts to do them both at the same time, the anaconda installer most often gets things mixed up and the system eventually doesn’t boot properly. Hope this helps if you’re still interested in the unique but highly effective combination of XCP-NG and Rockstor! Feel free to reach out if you have any questions. Take care.

5 Likes