I installed rockstor on my gen 10 today. There were a few issues - the bios has to be in legacy mode, but once it was the install was fine. Installing in uefi mode got me the error above.
Once installed there are a few kernel flags to prevent warnings ( around iommu, even though it is off in the bios - https://support.hpe.com/hpsc/doc/public/display?docId=emr_na-a00022034en_us Has the details) and for some reason I had to blacklist the i2c_designware_platform and i2c_designware_core kernel modules - in 4.12 they caused a kernel panic on startup, with 4.10 they just give a stacktrace on startup.
That sounds worse than it is though - the changes take 10 mins to make and the server now seems totally happy!