Duplicate key value violates unique constraint storageadmin_pool_name_key

[lakshmipathi_g]

I installed Rockstor and while access ‘Storage->Disks’ I get

No disks added. Click on Rescan to discover disks"

message and Rescanning gives below error.

"duplicate key value violates unique constraint “storageadmin_pool_name_key"
DETAIL: Key (name)=(rockstor_rockstor) already exists.”

During installation I selected ‘custom partition’ and /boot (300MB) and / (40GB).
Rockstor Minimum system requirements says:

One or more additional hard drives for data

I didn’t follow-above step during installation process. After booting into Dashboard - attached 4GB btrfs drive - Hoping that re-scan will detect this drive. Any thoughts how should I proceed?

[13/Nov/2015 13:40:38] DEBUG [smart_manager.data_collector:283] Sysinfo has connected
[13/Nov/2015 13:40:38] ERROR [smart_manager.data_collector:358] Failed to update disk state.. exception: OauthApp matching query does not exist.
[13/Nov/2015 13:40:38] ERROR [smart_manager.data_collector:358] Failed to update pool state.. exception: OauthApp matching query does not exist.
[13/Nov/2015 13:40:38] ERROR [smart_manager.data_collector:358] Failed to update share state.. exception: OauthApp matching query does not exist.
[13/Nov/2015 13:40:38] ERROR [smart_manager.data_collector:358] Failed to update snapshot state.. exception: OauthApp matching query does not exist.
[13/Nov/2015 13:40:38] DEBUG [smart_manager.data_collector:338] failed to update Rock-on metadata. low-level exception: OauthApp matching query does not exist.
[13/Nov/2015 13:41:30] ERROR [storageadmin.util:38] request path: /api/disks/scan method: POST data: <QueryDict: {}>
[13/Nov/2015 13:41:30] ERROR [storageadmin.util:39] exception: duplicate key value violates unique constraint "storageadmin_pool_name_key"
DETAIL:  Key (name)=(rockstor_rockstor) already exists.

[suman (suman chakravartula)]

You are right that disks can be added any time and rescan should detect them. The problem you ran into is happening because there are two pools with the same name(rockstor_rockstor). Perhaps it has something to do with custom partitioning. I plan to fix this as part of this issue.

In the mean time, the workaround I suggest in your case is, after install and before setup – run btrfs fi show and if there are to filesystems with the same rockstor_rockstor label, delete the one from previous install. Then run setup and you should not get this error. Hope this helps. Let us know.

[lakshmipathi_g]

Thanks for the prompt response, will try this and get back with its results

[lakshmipathi_g]

Re-installed again with custom partition. Verified there’s no duplicate label named ‘rockstor_rockstor’ with ‘btrfs fi show’ . After filling-up the user-data on setup page, I didnt get the dashboard but received error message ‘POST /api/disks/scan’ . Here’s the log:

# btrfs  fi show
Label: 'rockstor_rockstormyi2'  uuid: f46a639d-b0cd-4aa5-8af1-3ec580c31999
	Total devices 1 FS bytes used 1.37GiB
	devid    1 size 13.49GiB used 4.04GiB path /dev/sda7

Label: 'rockstor_rockstormy0'  uuid: ed3ea3da-4ce5-4120-acfd-9327592fead4
	Total devices 1 FS bytes used 224.00KiB
	devid    1 size 3.72GiB used 417.38MiB path /dev/sda6


[14/Nov/2015 11:59:19] DEBUG [smart_manager.data_collector:414] Listening on port http://127.0.0.1:8080 and on port 10843 (flash policy server)
[14/Nov/2015 12:01:43] DEBUG [smart_manager.data_collector:279] Sysinfo has been initialized
[14/Nov/2015 12:01:43] DEBUG [smart_manager.data_collector:283] Sysinfo has connected
[14/Nov/2015 12:01:43] ERROR [smart_manager.data_collector:358] Failed to update disk state.. exception: OauthApp matching query does not exist.
[14/Nov/2015 12:01:43] ERROR [smart_manager.data_collector:358] Failed to update pool state.. exception: OauthApp matching query does not exist.
[14/Nov/2015 12:01:43] ERROR [smart_manager.data_collector:358] Failed to update share state.. exception: OauthApp matching query does not exist.
[14/Nov/2015 12:01:43] ERROR [smart_manager.data_collector:358] Failed to update snapshot state.. exception: OauthApp matching query does not exist.
[14/Nov/2015 12:01:43] DEBUG [smart_manager.data_collector:338] failed to update Rock-on metadata. low-level exception: OauthApp matching query does not exist.
[14/Nov/2015 12:02:39] ERROR [storageadmin.util:38] request path: /api/disks/scan method: POST data: <QueryDict: {}>
[14/Nov/2015 12:02:39] ERROR [storageadmin.util:39] exception: duplicate key value violates unique constraint "storageadmin_pool_name_key"
DETAIL:  Key (name)=(rockstor_rockstor) already exists.
Traceback (most recent call last):
  File "/opt/rockstor/src/rockstor/rest_framework_custom/generic_view.py", line 40, in _handle_exception
    yield
  File "/opt/rockstor/src/rockstor/storageadmin/views/disk.py", line 107, in post
    return self._update_disk_state()
  File "/opt/rockstor/eggs/Django-1.6.2-py2.7.egg/django/db/transaction.py", line 339, in inner
    return func(*args, **kwargs)
  File "/opt/rockstor/src/rockstor/storageadmin/views/disk.py", line 74, in _update_disk_state
    p.save()
  File "/opt/rockstor/eggs/Django-1.6.2-py2.7.egg/django/db/models/base.py", line 545, in save
    force_update=force_update, update_fields=update_fields)
  File "/opt/rockstor/eggs/Django-1.6.2-py2.7.egg/django/db/models/base.py", line 573, in save_base
    updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
  File "/opt/rockstor/eggs/Django-1.6.2-py2.7.egg/django/db/models/base.py", line 654, in _save_table
    result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
  File "/opt/rockstor/eggs/Django-1.6.2-py2.7.egg/django/db/models/base.py", line 687, in _do_insert
    using=using, raw=raw)
  File "/opt/rockstor/eggs/Django-1.6.2-py2.7.egg/django/db/models/manager.py", line 232, in _insert
    return insert_query(self.model, objs, fields, **kwargs)
  File "/opt/rockstor/eggs/Django-1.6.2-py2.7.egg/django/db/models/query.py", line 1511, in insert_query
    return query.get_compiler(using=using).execute_sql(return_id)
  File "/opt/rockstor/eggs/Django-1.6.2-py2.7.egg/django/db/models/sql/compiler.py", line 899, in execute_sql
    cursor.execute(sql, params)
  File "/opt/rockstor/eggs/Django-1.6.2-py2.7.egg/django/db/backends/util.py", line 53, in execute
    return self.cursor.execute(sql, params)
  File "/opt/rockstor/eggs/Django-1.6.2-py2.7.egg/django/db/utils.py", line 99, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/opt/rockstor/eggs/Django-1.6.2-py2.7.egg/django/db/backends/util.py", line 53, in execute
    return self.cursor.execute(sql, params)
IntegrityError: duplicate key value violates unique constraint "storageadmin_pool_name_key"
DETAIL:  Key (name)=(rockstor_rockstor) already exists.

[suman (suman chakravartula)]

That is strange as there is not rockstor_rockstor pool. Thanks for the logs. I’ll take a look as port of issue #921

[lakshmipathi_g]

okay will wait for the #921 fix and then try again.

[alazare619 ("AL")]

This is affecting me on the current branch fully updated on testing updates with 2 1.5 TB seagates. and a small boot drive. I will update this post shortly with error log when I get back to the machine.

[alazare619 ("AL")]

duplicate key value violates unique constraint “storageadmin_disk_name_key” DETAIL: Key (name)=(scsi-S_ST31500341AS) already exists.

        Traceback (most recent call last):

File “/opt/rockstor/src/rockstor/rest_framework_custom/generic_view.py”, line 40, in _handle_exception
yield
File “/opt/rockstor/src/rockstor/storageadmin/views/disk.py”, line 272, in post
return self._update_disk_state()
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/utils/decorators.py”, line 145, in inner
return func(*args, **kwargs)
File “/opt/rockstor/src/rockstor/storageadmin/views/disk.py”, line 221, in _update_disk_state
dob.save()
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/base.py”, line 734, in save
force_update=force_update, update_fields=update_fields)
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/base.py”, line 762, in save_base
updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/base.py”, line 846, in _save_table
result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/base.py”, line 885, in _do_insert
using=using, raw=raw)
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/manager.py”, line 127, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/query.py”, line 920, in _insert
return query.get_compiler(using=using).execute_sql(return_id)
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/models/sql/compiler.py”, line 974, in execute_sql
cursor.execute(sql, params)
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/backends/utils.py”, line 64, in execute
return self.cursor.execute(sql, params)
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/utils.py”, line 98, in exit
six.reraise(dj_exc_type, dj_exc_value, traceback)
File “/opt/rockstor/eggs/Django-1.8.16-py2.7.egg/django/db/backends/utils.py”, line 64, in execute
return self.cursor.execute(sql, params)
IntegrityError: duplicate key value violates unique constraint "storageadmin_disk_name_key"
DETAIL: Key (name)=(scsi-S_ST31500341AS) already exists.

[phillxnet (Philip Guyton)]

@alazare619 First off welcome to Rockstor.

Your problem does look similar to the now closed referenced issue but is actually different.

From a quick look at the error message I would initially say that your 2 drives have the same serial number of “S” and Rockstor requires that each drive have a unique serial number. This may well be something else but that’s what it looks like initially. Are you using some kind of hyper visor, if so could you ensure it’s passing through serial numbers from the drives through correctly.

Also could you paste into this thread the output from the following commands:

ls -la /dev/disk/by-id/

and also:

lsblk -P -o NAME,MODEL,SERIAL,SIZE,TRAN,VENDOR,HCTL,TYPE,FSTYPE,LABEL,UUID

If you could also put a ``` on the preceeding and following lines of each paste that will make it easier to read.

Thanks. It’s a strange one this and obviously shouldn’t happen and I’m not sure yet why it is happening but that drive name does look suspicious as the ‘S’ is located where there would normally be a unique serial and Rockstor has it’s pre-requisite unique serials in order to track drives even without an fs on them. Let see what those commands output looks like and we can take it from there.

Thanks for the report and the logs.

[alazare619 ("AL")]

Yes I am using hyper-v I’ve attempted both IDE bus wich you can only have 2 IDE controllers and 2 on each boss 4 drives total…and SCSI Controller both with direct drive passthrough…This is just a dry run I’d like to migrate to this for some stuff and use the HBA and DAS SAS enclosure eventually to the guest VM but this is my first real problem with drive passthrough.

IDE Bus “ls -la /dev/disk/by-id/”

[root@rockstor ~]# ls -la /dev/disk/by-id/
total 0
drwxr-xr-x 2 root root 280 Dec 22 10:50 .
drwxr-xr-x 8 root root 160 Dec 22 10:50 ..
lrwxrwxrwx 1 root root   9 Dec 22 10:50 ata-Virtual_CD -> ../../sr0
lrwxrwxrwx 1 root root   9 Dec 22 10:49 scsi-3600224800718a8dcd346e05aca5af0cb -> ../../sda
lrwxrwxrwx 1 root root  10 Dec 22 10:49 scsi-3600224800718a8dcd346e05aca5af0cb-part1 -> ../../sda1
lrwxrwxrwx 1 root root  10 Dec 22 10:49 scsi-3600224800718a8dcd346e05aca5af0cb-part2 -> ../../sda2
lrwxrwxrwx 1 root root  10 Dec 22 10:49 scsi-3600224800718a8dcd346e05aca5af0cb-part3 -> ../../sda3
lrwxrwxrwx 1 root root   9 Dec 22 10:50 scsi-S_ST31500341AS -> ../../sdb
lrwxrwxrwx 1 root root  10 Dec 22 10:49 scsi-S_ST31500341AS-part1 -> ../../sdc1
lrwxrwxrwx 1 root root  10 Dec 22 10:49 scsi-S_ST31500341AS-part2 -> ../../sdc2
lrwxrwxrwx 1 root root   9 Dec 22 10:49 wwn-0x600224800718a8dcd346e05aca5af0cb -> ../../sda
lrwxrwxrwx 1 root root  10 Dec 22 10:49 wwn-0x600224800718a8dcd346e05aca5af0cb-part1 -> ../../sda1
lrwxrwxrwx 1 root root  10 Dec 22 10:49 wwn-0x600224800718a8dcd346e05aca5af0cb-part2 -> ../../sda2
lrwxrwxrwx 1 root root  10 Dec 22 10:49 wwn-0x600224800718a8dcd346e05aca5af0cb-part3 -> ../../sda3

IDE Bus “lsblk -P -o NAME,MODEL,SERIAL,SIZE,TRAN,VENDOR,HCTL,TYPE,FSTYPE,LABEL,UUID”

[root@rockstor ~]# lsblk -P -o NAME,MODEL,SERIAL,SIZE,TRAN,VENDOR,HCTL,TYPE,FSTYPE,LABEL,UUID
NAME="sdb" MODEL="ST31500341AS    " SERIAL="" SIZE="1.4T" TRAN="" VENDOR="        " HCTL="3:0:0:0"                TYPE="disk" FSTYPE="btrfs" LABEL="test" UUID="4ede2d74-72fe-4a09-add2-b6c2da36e32d"
NAME="sr0" MODEL="Virtual CD/ROM  " SERIAL="" SIZE="1024M" TRAN="ata" VENDOR="Msft    "     HCTL="0:0:1:0" TYPE="rom" FSTYPE="" LABEL="" UUID=""
NAME="fd0" MODEL="" SERIAL="" SIZE="4K" TRAN="" VENDOR="" HCTL="" TYPE="disk" FSTYPE=""     LABEL="" UUID=""
NAME="sdc" MODEL="ST31500341AS    " SERIAL="" SIZE="1.4T" TRAN="" VENDOR="        " HCTL="4:0:1:0"     TYPE="disk" FSTYPE="" LABEL="" UUID=""
NAME="sdc2" MODEL="" SERIAL="" SIZE="1.4T" TRAN="" VENDOR="" HCTL="" TYPE="part" FSTYPE="ntfs"     LABEL="New Volume" UUID="24B4FEE0B4FEB400"
NAME="sdc1" MODEL="" SERIAL="" SIZE="128M" TRAN="" VENDOR="" HCTL="" TYPE="part" FSTYPE=""     LABEL="" UUID=""
NAME="sda" MODEL="Virtual Disk    " SERIAL="600224800718a8dcd346e05aca5af0cb" SIZE="32G" TRAN=""     VENDOR="Msft    " HCTL="2:0:0:0" TYPE="disk" FSTYPE="" LABEL="" UUID=""
NAME="sda2" MODEL="" SERIAL="" SIZE="2G" TRAN="" VENDOR="" HCTL="" TYPE="part" FSTYPE="swap"     LABEL="" UUID="6feaa12d-95d9-4356-bf9f-8b466be6940b"
NAME="sda3" MODEL="" SERIAL="" SIZE="29.5G" TRAN="" VENDOR="" HCTL="" TYPE="part" FSTYPE="btrfs"     LABEL="rockstor_rockstor" UUID="f44bbe3d-6b5a-47e0-82e8-2bb4ac85708c"
NAME="sda1" MODEL="" SERIAL="" SIZE="500M" TRAN="" VENDOR="" HCTL="" TYPE="part" FSTYPE="ext4"     LABEL="" UUID="12b18dde-80cf-4991-9f11-10142aa5a287"
[root@rockstor ~]#

SCSI instead of IDE “ls -la /dev/disk/by-id/”

[root@rockstor ~]# ls -la /dev/disk/by-id/
total 0
drwxr-xr-x 2 root root 280 Dec 22 11:33 .
drwxr-xr-x 8 root root 160 Dec 22  2016 ..
lrwxrwxrwx 1 root root   9 Dec 22 11:33 ata-Virtual_CD -> ../../sr0
lrwxrwxrwx 1 root root   9 Dec 22 11:33 scsi-3600224800718a8dcd346e05aca5af0cb -> ../../sda
lrwxrwxrwx 1 root root  10 Dec 22 11:33 scsi-3600224800718a8dcd346e05aca5af0cb-part1 -> ../../sda1
lrwxrwxrwx 1 root root  10 Dec 22 11:33 scsi-3600224800718a8dcd346e05aca5af0cb-part2 -> ../../sda2
lrwxrwxrwx 1 root root  10 Dec 22 11:33 scsi-3600224800718a8dcd346e05aca5af0cb-part3 -> ../../sda3
lrwxrwxrwx 1 root root   9 Dec 22 11:33 scsi-S_ST31500341AS -> ../../sdc
lrwxrwxrwx 1 root root  10 Dec 22 11:33 scsi-S_ST31500341AS-part1 -> ../../sdc1
lrwxrwxrwx 1 root root  10 Dec 22 11:33 scsi-S_ST31500341AS-part2 -> ../../sdc2
lrwxrwxrwx 1 root root   9 Dec 22 11:33 wwn-0x600224800718a8dcd346e05aca5af0cb -> ../../sda
lrwxrwxrwx 1 root root  10 Dec 22 11:33 wwn-0x600224800718a8dcd346e05aca5af0cb-part1 -> ../../sda1
lrwxrwxrwx 1 root root  10 Dec 22 11:33 wwn-0x600224800718a8dcd346e05aca5af0cb-part2 -> ../../sda2
lrwxrwxrwx 1 root root  10 Dec 22 11:33 wwn-0x600224800718a8dcd346e05aca5af0cb-part3 -> ../../sda3

SCSI Instaed of IDE “Fdisk -l”

[root@rockstor ~]# fdisk -l
Disk /dev/sda: 34.4 GB, 34359738368 bytes, 67108864 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk label type: dos
Disk identifier: 0x000f0524

Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048     1026047      512000   83  Linux
/dev/sda2         1026048     5222399     2098176   82  Linux swap / Solaris
/dev/sda3         5222400    67108863    30943232   83  Linux

Disk /dev/sdb: 1500.3 GB, 1500301910016 bytes, 2930277168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk /dev/sdc: 1500.3 GB, 1500301910016 bytes, 2930277168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x00000000

Device Boot      Start         End      Blocks   Id  System
/dev/sdc1               1  4294967295  2147483647+  ee  GPT

SCSI Instead of IDE “lsblk -P -o NAME,MODEL,SERIAL,SIZE,TRAN,VENDOR,HCTL,TYPE,FSTYPE,LABEL,UUID”

[root@rockstor ~]# lsblk -P -o NAME,MODEL,SERIAL,SIZE,TRAN,VENDOR,HCTL,TYPE,FSTYPE,LABEL,UUID
NAME="sdb" MODEL="ST31500341AS    " SERIAL="" SIZE="1.4T" TRAN="" VENDOR="        " HCTL="3:0:0:0"     TYPE="disk" FSTYPE="btrfs" LABEL="test" UUID="4ede2d74-72fe-4a09-add2-b6c2da36e32d"
NAME="sr0" MODEL="Virtual CD/ROM  " SERIAL="" SIZE="1024M" TRAN="ata" VENDOR="Msft    " HCTL="1:0:0:0" TYPE="rom" FSTYPE="" LABEL="" UUID=""
NAME="fd0" MODEL="" SERIAL="" SIZE="4K" TRAN="" VENDOR="" HCTL="" TYPE="disk" FSTYPE="" LABEL="" UUID=""
NAME="sdc" MODEL="ST31500341AS    " SERIAL="" SIZE="1.4T" TRAN="" VENDOR="        " HCTL="3:0:0:1" TYPE="disk" FSTYPE="" LABEL="" UUID=""
NAME="sdc2" MODEL="" SERIAL="" SIZE="1.4T" TRAN="" VENDOR="" HCTL="" TYPE="part" FSTYPE="ntfs" LABEL="New Volume" UUID="24B4FEE0B4FEB400"
NAME="sdc1" MODEL="" SERIAL="" SIZE="128M" TRAN="" VENDOR="" HCTL="" TYPE="part" FSTYPE="" LABEL="" UUID=""
NAME="sda" MODEL="Virtual Disk    " SERIAL="600224800718a8dcd346e05aca5af0cb" SIZE="32G" TRAN="" VENDOR="Msft    " HCTL="2:0:0:0" TYPE="disk" FSTYPE="" LABEL="" UUID=""
NAME="sda2" MODEL="" SERIAL="" SIZE="2G" TRAN="" VENDOR="" HCTL="" TYPE="part" FSTYPE="swap" LABEL="" UUID="6feaa12d-95d9-4356-bf9f-8b466be6940b"
NAME="sda3" MODEL="" SERIAL="" SIZE="29.5G" TRAN="" VENDOR="" HCTL="" TYPE="part" FSTYPE="btrfs" LABEL="rockstor_rockstor" UUID="f44bbe3d-6b5a-47e0-82e8-2bb4ac85708c"
NAME="sda1" MODEL="" SERIAL="" SIZE="500M" TRAN="" VENDOR="" HCTL="" TYPE="part" FSTYPE="ext4"     LABEL="" UUID="12b18dde-80cf-4991-9f11-10142aa5a287"

[phillxnet (Philip Guyton)]

@alazare619 Thanks for the outputs.

Very much looks like a serial issue as 2 drives are being mapped over the top of one another by udev, which is below Rockstor’s function but is required by Rockstor to be able to discern drives and their details.

As you see from the by-id listing there is only one base drive listed “sdb” yet “sdc” has partitions that “sdb” doesn’t and so they get mapped against the same drive by-id name.

lrwxrwxrwx 1 root root   9 Dec 22 10:50 scsi-S_ST31500341AS -> ../../sdb
lrwxrwxrwx 1 root root  10 Dec 22 10:49 scsi-S_ST31500341AS-part1 -> ../../sdc1
lrwxrwxrwx 1 root root  10 Dec 22 10:49 scsi-S_ST31500341AS-part2 -> ../../sdc2

Ie we have no base device listing for “sdc” in the above by-id listing. Probably because the name was already taken by sdb’s base device. But we know they are both there as lsblk lists them both and we can see there the difference in their partitions. Also note that if lsblk can’t retrieve serial numbers, as is the case here. Then Rockstor will fail over to using udev, however udev can’t cope with the non unique serials and hence the breakage.

The command Rockstor uses to find a drives serial if lsblk can’t see it is:

udevadm info --name=sdb

and then it parses the various serial outputs presented there to arrive at the same as what lsblk would have output were it to be able to retrieve the serial.

Udev uses serial numbers to tell drives apart and give them unique names, here sdb and sdc (the temp-names for your ST31500341AS model drives), but given your hyper-v is giving them both the identical serial number of “S” you have a broken client machine. At least as far as udev and consequently Rockstor is concerned.

Hope that helps.

[alazare619 ("AL")]

https://hastebin.com/comiqahuci.coffeescript

There is fdisk lsblk an attemp to spoof the serial which does seem to work kinda atleast the webui sees a diffrent serial now for each but it says the device key name is not unique now…

duplicate key value violates unique constraint “storageadmin_disk_name_key” DETAIL: Key (name)=(scsi-S_ST31500341AS) already exists.

Screenshot of my webui now. http://pasteboard.co/die9ReYKV.png

[phillxnet (Philip Guyton)]

@alazare619 Yes I wouldn’t go this way as the rules you used are based off the temp-names from what I can tell which was the whole point in moving away from temp names to something that moved with the actual device, not the changable sda type names. Also there are an increasing number of things that depend on the by-id name that will not work if you have 2 drives essentially fighting for the same by-id base name. The name is an artifact of udev rules itself and they depend the uniqueness and existence of the device serials. Your issue is with the hyper visor software not the client OS: that is where it should be fixed. The by-id name, derived from the serial, is meant to be unique and isn’t. A prerequisites of scsi compliant devices:

See RedHats: Persistent Naming.

"The World Wide Identifier (WWID) can be used in reliably identifying devices. It is a persistent, system-independent ID that the SCSI Standard requires from all SCSI devices. The WWID identifier is guaranteed to be unique for every storage device, and independent of the path that is used to access the device.

This identifier can be obtained by issuing a SCSI Inquiry to retrieve the Device Identification Vital Product Data (page 0x83) or Unit Serial Number (page 0x80). The mappings from these WWIDs to the current /dev/sd names can be seen in the symlinks maintained in the /dev/disk/by-id/ directory."

Hope that helps.

This is not something Rockstor can fix as the VM machine you are running it in is not creating devices that are friendly to udev and Rockstor uses udev heavily. You are likely to risk data loss if drives cannot be uniquely identified, hence udev and Rockstor’s assumption that each drive will have a unique serial. How else does one distinguish between and track devices when they transition between having no partition table or any fs. This is the challenge that udev and consequently Rockstor face and is only really doable by depending on serial numbers being unique: hence udev creating a naming scheme in by-id that utilises serial numbers.

When you find out how to fix / configure the hyper visor in this regard it would be good if you could post the same here as @unomagan did in the following thread with regard to Vmware which just plain give no serial (and hence no by-id name either), but yours is giving out the same “S” or “S_ST31500341AS” (from the udev command) serial to 2 different devices and so surfacing differently:

VMWare giving: ['ERROR: not a valid btrfs filesystem: /dev/disk/by-id/sda3', '']

Thanks! found it! While the error is very very misleading though… For anyone interested: disk.EnableUUID=“TRUE” added / Edited in the VMX file of the VM

I’m assuming there must be some counterpart to your chosen VM manager.

Apologies if I’ve missed something here but it currently looks pretty clear cut and a quick config in the hyper visor and you should be good. Otherwise you are on shaky ground with regard to udev on any linux.

Let us know how you get on as I at least haven’t seen this particular serial issue before so knowing the appropriate config would be good for others that come after you on the forum.

Thanks.

[alazare619 ("AL")]

@phillxnet Where does the Name Column come from. According to what i’m reading in this error message its no longer complaining about serial per say because they are now VDP-1500-1 and VDP-1500-2 (Also my hyper visor the slots never change so I’m not to worried)

The column has the below.

Error Message Stating its Key Name which is the Name column.

[phillxnet (Philip Guyton)]

@alazare619 Hello again.
The device names are generated by CenOS 7’s udev rules and Rockstor then matches the temp lsblk names to the boot safe by-id names in /dev/disk/by-id by working backwards from the sda type names on each scan; but tracks each device by serial in the db. You would also have to enhance your udev work around (which I think is the wrong way to go, but I get what you are saying, it’s just a risky route) to also correctly generate unique by-id names. In other words your are going to have to also edit the existing default udev rules for block devices to work as they should given your serial attribution hack backed up by temp names (although yes you say they are stabel). This will be completely dependant on those sda type names you are generating the serials from being static. If they move it will mess everything up. Also you will have to follow the expected format for these names, hence modifying the existing ones. It really would be better to sort this at the proper level that has failed and as such I can’t really help more in this direction.

As for Rockstor’s Device management please see:

Device management in Rockstor

This is a wikified post documenting how Rockstor manages devices and is intended to serve as a live document on the same; user documentation is available at Disks. Please update this developer document when you make any changes to the disk management subsystem within Rockstor’s code. Also any suggestions on how this might be done better are very welcome; but please make sure that these suggestions are clearly indicated as such; this helps to maintain the relationship between this document and th…

My final though it that you are doing it wrong and will end up shooting yourself in the foot and taking far too much time to get what is essentially a poor mock up that won’t tell you how Rockstor works as you are undermining the assumption of hardware that udev makes, let along Rockstor. Yet a single config for each drive would sort this at the hyper visor level.

However your rules are cute and if those names never move (which they almost always do) then you have supposedly unique serials, you just also then need unique names generated in /dev/disk/by-id based on those serials hence the route to alter the default udev rules.

Hope that helps and that you see that I see you are bending over backwards to hack around a misconfigured hyper-v. Or is it that this is something that just isn’t possible with hyper-v?

Maybe someone who is more familiar with hyper-v can step in and help on the hdd serial pass through / attribution bit as then you are sorted. Also my decades of battling with MS OS approximations are over and will remain that way.

Good luck.

[alazare619 ("AL")]

@phillxnet As far as I can tell documentation is sparse about passing the device serial number through hyper-v DOM0 to the hyper-v Guest…Hyper-V says you shouldn’t use passthrough and instead should make a fixed VHDX that is given to the guest…I just find that rather padantic to be frank.

[phillxnet (Philip Guyton)]

@alazare619 Yes, I did have a little search but couldn’t find anything. But with a little time someone may pop in on this thread with some pointers as I’m just not familiar with it at all I’m afraid.

[alazare619 ("AL")]

Ended up testing with fixed VHDX size everything looked good moved to a live mockup and so far happy any idea when iScsi will be available?

[phillxnet (Philip Guyton)]

@alazare619 That’s great. Thanks for the update and glad it’s going well.

As far as iSCSI is concerned we do have an issue open for it, but I’m not aware of any timing:

Issue: support iSCSI

But it’s not in our current milestone of Pinnacles.