Get() returned more than one RockOn -- it returned 2!

aymenSo · May 4, 2023, 4:14pm

Hello,

Sorry for the poor problem description, first time posting.
thank you for your time!

Brief description of the problem

Rock-ons Update return an error message

Detailed step by step instructions to reproduce the problem

Created custom rock-ons json file
update rock-ons
install custom rock-ons OK
remove rock-ons OK
delete json files
error message

Web-UI screenshot

Error Traceback provided on the Web-UI


            Traceback (most recent call last):
  File "/opt/rockstor/src/rockstor/rest_framework_custom/generic_view.py", line 41, in _handle_exception
    yield
  File "/opt/rockstor/src/rockstor/storageadmin/views/rockon.py", line 128, in post
    self._delete_deprecated(rockons)
  File "/opt/rockstor/.venv/lib/python2.7/site-packages/django/utils/decorators.py", line 185, in inner
    return func(*args, **kwargs)
  File "/opt/rockstor/src/rockstor/storageadmin/views/rockon.py", line 153, in _delete_deprecated
    RockOn.objects.get(name=cr).delete()
  File "/opt/rockstor/.venv/lib/python2.7/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/opt/rockstor/.venv/lib/python2.7/site-packages/django/db/models/query.py", line 384, in get
    (self.model._meta.object_name, num)
MultipleObjectsReturned: get() returned more than one RockOn -- it returned 2!

Hooverdan · May 4, 2023, 4:49pm

@aymenSo welcome to the Rockstor community.
Just for reference, which version of Rockstor are you running?
Couple of questions:

The custom rockon you installed (via the rockons-metastore directory), were those variations on existing Rockons, or brand-new ones?
Also, when you mention “remove” are you referring to using the uninstall button on the Webui, or did you also run the console command

poetry run delete-rockon <rockon-name>

And after that removed the json files?

aymenSo · May 5, 2023, 12:28pm

thank you @Hooverdan ! Here are my answers:

I am running Rockstor 4.5.6-0

Custom rockons are a mix of variations on existing Rockons + brand-new ones

Mainly I use the uninstall button on the Webui to “remove” Rockons but also used command line
for some Rockons where uninstall button was unsuccessful.

Yes, I removed json files

phillxnet · May 5, 2023, 12:52pm

@aymenSo Hello here. Not directly relevant but give you are on our testing channel, you may as well be using the latest version re:

See:

Shouldn’t affect your issue but it’s best to be reporting / experimenting against the latest code: just in case.

Also be sure to press the Rock-ons update button after making any changes. Expecially with custom Rock-on json files etc. And be carefull to use different names for these custom Rock-ons. It looks like you have had a clash of names from the following part of the message:

@Hooverdan and @Flox are more knowledgable on the Rock-on system. But if anyone can chip in on this observed failure then great. We can always improve the robustness of such custom experiments.

Not of much help I realise but just wanted to chip in on this one re use of latest testing, if testing.

aymenSo · May 5, 2023, 2:13pm

@phillxnet thank you!
Actually updated to Rockstor 4.5.9-1 + server reboot.
All seems working good.
Still have the same error message when I click on “Update” Rock-ons.
Here is a view of “All” Rock-ons page , you can clearly see 2 times the same rockon Calibre.local:

I already uninstalled Calibre.local rockon and removed the corresponding json file but strill have the error message.
Thank you again for your help!

Flox · May 5, 2023, 2:28pm

Thanks for the details, @aymenSo

Can you confirm what happens, or does not happen, if you now run the delete-rockon script mentioned by @Hooverdan? It would be:

cd /opt/rockstor 
poetry run delete-rockon Calibre.local

And then clock again on the “Update” button in the Rock-Ons page.

aymenSo · May 5, 2023, 2:45pm

Thank you @Flox for your message!

here is the result of the command:

I still have the same error message for “Update” button after this command.

Flox · May 5, 2023, 2:54pm

Interesting… I don’t recall seeing that issue in the past.
Ok, this will need a little look into the state of your db, then. I unfortunately can’t guide you on that right now but will try to as soon as possible. I think we will need to get the id of one of the two instances of that problematic rockon and then manually delete its record.

Ok the upside, you unearthed an unlikely but obviously possible scenario for which we need to account in the rockon-delete script.

Sorry again I can’t be of more help at this moment.

Hooverdan · May 5, 2023, 3:06pm

yet another question from me, did you at any point have two json files in the ‘/opt/rockstor/rocksons-metastore’ directory where the container name was the same, e.g. calibreserver

    "containers": {
      "calibreserver": {
        "image": "linuxserver/calibre",
        "launch_order": 1,

Usually that would lead to an error during refresh already, when the official and custom Rockon json files are evaluated and not finish the processing, so just want to make sure that there is a new scenario (though shouldn’t be) that would allow to load essentially duplicate data into the database…

Edit –
the flip side of this. If you did not have the above situation, did you have 2 json files with different container names (see above) but the same Rockon name Calibre.local at any point?

Hooverdan · May 5, 2023, 4:12pm

As we probably need to enhance the deletion script at some point, I opened a corresponding Github issue for this, as we continue to explore how this scenario came about (and hopefully find a root cause at some point).

github.com/rockstor/rockstor-core

Custom Rockon management deletion script enhancement.

opened 04:08PM - 05 May 23 UTC

Hooverdan96

This issue is triggered by the experience of forum user `aymenso`: https://foru…m.rockstor.com/t/get-returned-more-than-one-rockon-it-returned-2/8790 the Rockon delete script should consider an inconsistency scenario, where the same rockon is represented twice in the database: this should not occur, however we should probably prepare for that case, while working towards avoiding this scenario (possibly occurring during the json ingestion step). https://github.com/rockstor/rockstor-core/blob/0249f5283203461d71093447611c762901f6f378/src/rockstor/scripts/rockon_delete.py#LL40C1-L57C1 May be the above scenario can be addressed by adding an outer loop over the `ro` object, so it would treat possible duplicates as individual rockons and delete accordingly? Additional observation While trying to reproduce the inconsistency (which I was not successful in), this is what I saw (all on 4.5.9-1): - taking example of scrutiny rockon. Create two custom json files, differentiated by file name (scrutiny1.json, scrutiny2.json) and `containers` name (scrutiny1 and scrutiny2), but having the same Rockon name: `Scrutiny.local`. I made a non-relevant (I think) change to the description to be able to differentiate them, everything else remained the same. - After a refresh, only one Rockon appeared, namely the scrutiny2.json's content. <img width="886" alt="image" src="https://user-images.githubusercontent.com/35113775/236507338-5e8d0a1e-84b5-46c9-83e3-402d19337cae.png"> - Without installing it, but directly going for the script for rockon deletion, it removed the custom scrutiny entry without error. - Redoing the setup. Now installing the visible custom Rockon - no issue <img width="1425" alt="image" src="https://user-images.githubusercontent.com/35113775/236508707-383677cd-907f-4aae-84d8-2aa80ba007fd.png"> - Refreshing the Rockon list, while the Scrutiny.local is installed. scrutiny1.json contents are not picked up, i.e. there is still only 1 Scrutiny.local (installed). - Stopping and uninstalling Scrutiny.local using the UI. No issue. Selecting refresh, still only 1 (scrutiny2.json) is visible in the list. - Reinstalling Scrutiny.local and while it's running use the delete script. Reload the web page, Scrutiny.local rockon has disappeared completely (as expected). Now Refresh Rockons again. scrutiny2.json is picked up as before. So, not sure really how this inconsistency can happen

Flox · May 6, 2023, 8:37pm

Hi @aymenSo ,

Let’s see if we can get a better idea of what you have here. It’ll require some back-and-forth unfortunately, but I’d like to take that opportunity to have as much information as we can so that we can cover this situation in our delete-rockon script. Given it seems very hard to reproduce despite all of @Hooverdan’s efforts, I’d like to not miss anything.

We would first need to see what your relevant database looks like. As mentioned above, this would be via manual queries against your database. In the command below, the password requested is rocky.

# psql -U rocky -d storageadmin -c "SELECT id,name,state,status,taskid FROM storageadmin_rockon WHERE name = 'Calibre.local';"

Hopefully this should give us a coherent first answer.

Jorma_Tuomainen · May 7, 2023, 2:35pm

I’m running my own json since in the past the rockon pull request took a long time to get in

Hooverdan · May 7, 2023, 3:25pm

Interesting. Another quick question from my side. The json file you’re using locally for Emby, is that the same as the official Rocklin one or did you make any changes to it?

aymenSo · May 7, 2023, 3:25pm

@Hooverdan Yes, I did his mistake and got an error message.

@Flox Here is the result of the command:

aymen@mynas:~> psql -U rocky -d storageadmin -c "SELECT id,name,state,status,taskid FROM storageadmin_rockon WHERE name = 'Calibre.local';"
Password for user rocky:
 id  |     name      |   state   |        status         | taskid
-----+---------------+-----------+-----------------------+--------
 110 | Calibre.local | available | exitcode: 137 error:  |
 109 | Calibre.local | available | stopped               |
(2 rows)

Flox · May 7, 2023, 3:38pm

Thanks @aymenSo and @Jorma_Tuomainen for the reports… very curious to see two reports of that peculiar case, especially given we have not touched the Rock-Ons side of things lately…

Let’s see what you have in terms of containers, then:
@aymenSo :

# psql -U rocky -d storageadmin -c "SELECT * FROM storageadmin_dcontainer WHERE rockon_id IN (109,110);"

@Jorma_Tuomainen:

# psql -U rocky -d storageadmin -c "SELECT * FROM storageadmin_dcontainer WHERE rockon_id IN (103,104);"

Jorma_Tuomainen · May 7, 2023, 8:21pm

postgres@rockstor:~> psql -d storageadmin -c “SELECT * FROM storageadmin_dcontainer WHERE rockon_id IN (103,104);”
id | rockon_id | dimage_id | name | launch_order | uid
-----±----------±----------±-----±-------------±----
112 | 104 | 68 | emby | 1 |
(1 row)

Flox · May 8, 2023, 1:25pm

Thanks, @Jorma_Tuomainen.

Only one of your Rock-On entries has a related container, then… It thus matches my test environment so you should be able to use the quick fix I tested to work in this case.
I unfortunately cannot upload not right now but should be able to later today.

Hope this helps.

Flox · May 8, 2023, 3:24pm

Just committed my changes so that you guys can try and run that updated version to reset these custom Rock-Ons. To try:

cd /opt/rockstor/src/rockstor/scripts
wget -O rockon_delete.py https://raw.githubusercontent.com/FroggyFlox/rockstor-core/1b3458fcb47313999242db58e73a5d994108bb4f/src/rockstor/scripts/rockon_delete.py

Then try to re-run the script:

cd /opt/rockstor
poetry run delete-rockon <Rockon-name>

It should list two deletions.

Hope this helps,

Jorma_Tuomainen · May 8, 2023, 3:59pm

Scratch that error, went back and noticed another rock on that had 2 entries (one failed install).

UI loads now, maybe make UI not crash and provide delete button there?

Hooverdan · May 8, 2023, 4:25pm

@Jorma_Tuomainen so the script worked for you then?
The UI suggestion could be a future option, I think @Flox (or all of us, collectively ) wants to figure out first how this inconsistent state can occur in the first place, since in the past (at least as far back as I can remember) this duplication has not happened before.