I plan to buy a new skylake-based desktop. Am I right that any hardware that is compatible with CentOS will work with Rockstor? It will be both a VM host for my lab and Rockstor NAS. I will install a bare-metal hypervisor, either XenServer or ESXi. It will have two x 6tb data disks and a small SSD for the hypervisor and Rockstor. Does the following process sound right?
- Install hypervisor on SSD
- Spin up Rockstor as the first VM on SSD.
- Create RAID-0 pool, share and NFS export on the data disks using Rockstor
- Create VMWare datastore or XenServer storage repository from the NFS export.
- Spin up lab VMs running on datastore\SR.
Is this a sensible plan, any dragons be here?
you meant raid1?
i have seen similar setups before, should work like a charm, you should use vtd for passing through, so you might want to add an extra controller or check that data and ssd disks are on different controllers for the motherboard.
Thanks for your suggestions felix. I neglected to add that eventually we will have a second NAS that this one will replicate to, so RAID-0 is fine.
“vtd for passing through”: I assume you mean if we add a dedicated raid controller for the SSD that has this enabled it will give the host & Rockstor VM a performance boost. SATA3 gives 6Gb\s (750MB\s). Two disks might throughput 250MB max and an SSD 500MB, so hopefully can do without this cost\complexity without too much of a performance hit.
However, I assume Rockstor will serve data to the hypervisor over a virtual NIC which will be limited to Gb LAN. Probably worthwhile adding a second or even a third virtual NIC to the Rockstor VM and bond them together to see whether this increases performance.
Damn, just seen network-bonding isn’t supported yet. I’ll keep a close eye on this issue, as will be great for setups like this.
for inter host <-> rockstor connections the vnic should support 10gbit, only external connections are limited by you physical cards
with vtd i meant exactly that, vtd enables your host hypervisor to directly pass through some hardware to some vm
without vtd your vm would use virtual disks which themselves are (somewhat) blockdevices on the host filesystem for that disk
with linux kvm you can “passthrough” a blockdevice itself, with vmware that is not possible, for xen i dont know
XenServer has passthrough too. Would you suggest passing through the disks or SSD to the Rockstor VM? The hypervisor will be installed on the SSD, so will enabling pass-through on the SSD make it dedicated to Rockstor and inaccessible to the hypervisor? Or am I misunderstanding how passthrough works.
Sounds like it’s worth a scoot around for a cheap raid-controller.
mostly any hypervisor has vtd support, but for vtd you would need a cpu that supports vtd and most importantly a mainboard that supports it as well
what i meant by “passthrough” in linux kvm is a pseudo passthrough without vtd where you just give the /dev/sdX device to the kvm machine.
yes, enabling passthrough for a device (not disk!) makes this device dedicated to the vm, so i would suggest using a plain virtual disk for the rockstor base system and passthrough of a hba to rockstor.
available devices for vtd might be (from one of my vmware servers):
if you have an pcie hba connected that would be in the list too
keep in mind that you need a hba or a flashed raid card where the raid card just passes through the disks without doing raid itself.
you can, ofc, use plain virtual disks (i think vmware upped the limit from 2tb to somewhat more) and use that multilayered device subsystem, but that adds overhead and isnt best practices but it would work
Yes I checked CPU & motherboard support vtd. Think I get it now, so new process would be:
- Install hypervisor on SSD.
- Spin-up Rockstor VM on SSD (plain virtual disk).
- Pass through HBA with two disks attached to Rockstor VM (along with any other devices I want to dedicate).
- As before create RAID pool, NFS export, etc…
If I get it all working, I’ll report back performance numbers with and without passthrough raid card. Thanks again.
yes thats correct, some performance numbers on the btrfs on virtual disk on host filesystem on hdd vs btrfs on hdd would be great