File permissions

jihiggs · May 17, 2023, 2:02pm

I am still not terribly clear on how linux file permissions work. I get the user/group/everyone aspect and how the numbers work, ie 777, 770, 760… heres what I have come up with…
I have several shares with coresponding smb, backups, blueiris (my surveilance recording software), media (movies and stuff like that), share (generic share that everyone can write to), and stuff (my collection of junk that only my primary computer and laptop should be able to access). ive got usernames for each of these purposes. in shares, I make the owner and group one of those usernames, usually 770 permissions. in the case of “share” I give it 777. how does this translate to smb shares where I can add a share administrator, and that user now seems to have full access? what in linux is changed to allow this? the file permissions dont seem to have changed to allow this.

on a related note, ive reinstalled rockstor tumbleweed, and enabled testing updates to get the most recent release candidate (did I get that right?). all my pools imported fine, and the shares were still intact, though all the permissions were reset to root/root. I created all my users, and modified the shares permisions to the appropriate user/user. then I created the smb shares and made my primary account admin of those shares so I can access all of them.

the problem is, one of my shares “backups”, when I try to modify shares/access control, I set it to backups/backups, 770. I can see in top, chown starts chugging away and the drives start jittering, then I get this:

I never see chmod come up. the other shares did not produce an error. ls -la on /mnt2/backups shows 770 and the uid of the backups user, not the name of the user for some reason? the sub directories show backups/backups but the permissions are inconsistent. I am now out of time, but I wanted to get eyes on this and hopefully suggestions. please let me know what log files if any you want to see. my plan is to manually set permissions to this folder in the terminal and see what happens. but ive made changes in terminal instead of the gui and it just seems to get me into trouble.

Hooverdan · May 17, 2023, 5:15pm

@jihiggs I can only share my own experience/approach when I ran into jumbled up ownership/permission setups across and within my various shares that I have.
The root cause in my scenario was that at some point I changed some Rockon users (e.g. in Plex that offers what user/group combo to use to run it, and by extension on how it would create/generate files).

I did immediately resort to the command line to rectify that since some changes were at different levels, and once that was done, the UI also showed the appropriate ownership/permissions (as it should). On one share with a lot of files/directories, the chown took quite a long time. The chmod I “reset” only in selected folders where it was important. On another I did a global chmod of existing files (i.e. using option -R) which also took a considerable amount of time.

In the logs, can you see any mention of an error related to the underlying chown running? If you were changing a lot of data using chown, may be you’re running into some timeout problem?

I suspect, @phillxnet might have to chime on possible underlying root causes.

jihiggs · May 17, 2023, 6:53pm

well I did a chmod -R 770 on the backups folder and the permissions are right, however the rockstor gui doesnt reflect the actual permissions of the files, and still shows the owner as root, which it isnt.

Hooverdan · May 17, 2023, 9:11pm

Interesting. I checked on one of my setups and noticed something similar. The Rock-on had modified (I believe) the ownership of that directory, however in the UI the original setup was still present:

Now, when I changed the owner (chown) via the UI, the terminal and UI output were in sync:

as for the permissions, those apparently remained the same as originally set:

server:/mnt2 # stat -c '%A %a %n' minecraft
drwxr-xr-x 755 minecraft

vs. UI

now, changing the permissions via terminal to a different value, say:
drwxrwxr-x 775 minecraft

the value remains the same as before in the UI (755) - as you have observed.

Now, when I change the permissions via the UI to, say, this:

now, via terminal you can see the change:
drwxr--r-- 744 minecraft

Without photographic proof I can also confirm that all sub-directories and files have that permission change applied.

So, I think, this means that during the initial setup of a share or changes of the attributes, that information is stored on the Rockstor database and applies to the root directory of the share. If no terminal-level meddling occurs, those would remain in sync.
However, if those owners and permissions are changed without going through the UI, they will go out of sync.

Looking at the code (without understanding everything) it seems that during creation a default owner/permission piece is set up (during initial share setup one cannot assign a owner/permission set).

github.com

rockstor/rockstor-core/blob/bdc3df2762ceddf355b482ca50f698654e342861/src/rockstor/storageadmin/views/share_acl.py#L31-L40


      
          def post(self, request, sid):
              with self._handle_exception(request):
                  share = Share.objects.get(id=sid)
                  options = {
                      "owner": "root",
                      "group": "root",
                      "perms": "755",
                      "orecursive": True,
                      "precursive": True,
                  }

After the initial creation of the share, one can go back in and make a change to the owner/permissions.

github.com

rockstor/rockstor-core/blob/bdc3df2762ceddf355b482ca50f698654e342861/src/rockstor/storageadmin/views/share_acl.py#L41-L53


      
          options["owner"] = request.data.get("owner", options["owner"])
          options["group"] = request.data.get("group", options["group"])
          options["perms"] = request.data.get("perms", options["perms"])
          options["orecursive"] = request.data.get(
              "orecursive", options["orecursive"]
          )
          options["precursive"] = request.data.get(
              "precursive", options["precursive"]
          )
          share.owner = options["owner"]
          share.group = options["group"]
          share.perms = options["perms"]
          share.save()

(overriding the previous default values)

Those will be recursively applied to the share (both ownership and permissions).

github.com

rockstor/rockstor-core/blob/bdc3df2762ceddf355b482ca50f698654e342861/src/rockstor/storageadmin/views/share_acl.py#L60-L62


      
          chown(mnt_pt, options["owner"], options["group"], options["orecursive"])
          chmod(mnt_pt, options["perms"], options["precursive"])
          if force_mount is True:

So, obviously in the UI, the display of that information comes from the database under which those UI values were saved. Terminal-based changes on the root share or sub-directories and files are not comprehended or queried. Here is the model, storing that information:

github.com

rockstor/rockstor-core/blob/bdc3df2762ceddf355b482ca50f698654e342861/src/rockstor/storageadmin/models/share.py#LL29-L45


      
          class Share(models.Model):
              # pool that this share is part of
              pool = models.ForeignKey(Pool)
              """auto created 0/x qgroup"""
              qgroup = models.CharField(max_length=100)
              """quota group y/x explicitly created for this Share"""
              pqgroup = models.CharField(max_length=32, default=settings.MODEL_DEFS["pqgroup"])
              """name of the share, kind of like id"""
              name = models.CharField(max_length=4096, unique=True)
              """id of the share. numeric in case of btrfs"""
              uuid = models.CharField(max_length=100, null=True)
              """total size in KB"""
              size = models.BigIntegerField(default=0)
              owner = models.CharField(max_length=4096, default="root")
              group = models.CharField(max_length=4096, default="root")
              perms = models.CharField(max_length=9, default="755")
              toc = models.DateTimeField(auto_now=True)

So, in summary, you are right, the UI view and what you have in the share can be out of sync, once the terminal becomes involved. I also believe, that the forced recursion of any changes via the UI (Owner and/or Permissions) can time out when it needs to be applied to a large set of directories/files that already exist on that share…
So, for better or worse, whatever you do with the terminal is the one the system honors. How to get the UI entries realigned or whether that’s even a desirable functionality, since the terminal use is not a standard use case, is worth a discussion.

jihiggs · May 17, 2023, 10:37pm

new information:
the owner of the file was 3001, I mistakenly thought that was one of the users I created, it was a user on the previous install. in terminal I was still unable to chown the directory. it had an immutable flag, I removed it with sudo chattr -i backups. still doesnt show the correct owner or permissions in the gui, but I can now access the file without making the user an admin of the smb share.

Hooverdan · May 17, 2023, 11:02pm

good deal.

I guess, if you want them to be in sync again, you have to go into the UI, adjust the owner/permissions and save - it will trigger the chown/chmod again on that share, but then you would be in sync.

jihiggs · May 17, 2023, 11:10pm

tried, it didnt work even after removing the immutable flag

Hooverdan · May 17, 2023, 11:10pm

And for completeness sake, the immutable issue was observed in the past before (similar and different scenarios like yours);

and there is an associated documentation update logged on Github to point out the checking of ownerships when importing, before proceeding.

github.com/rockstor/rockstor-doc

add note to check ownership of shares post import

opened 11:22AM - 12 May 22 UTC

phillxnet

Thanks to forum member parth for highlighting this issue. Imported pools can end… up with root:root ownership on previously edited user/group shares (subvols). It is propose we add a note to this end to inform users to expect and check for this outcome. Relates to the following major doc section: https://rockstor.com/docs/interface/storage/disks.html#import-btrfs-pool Form thread reference: https://forum.rockstor.com/t/solved-btrfs-subvolume-create-delete-fail/3943/11 We may also have a bug here regarding restoring but for now a doc entry would be a good heads up.

Hooverdan · May 18, 2023, 2:34pm

are you then still getting the error you first reported when you try it via the WebUI?

jihiggs · May 19, 2023, 1:24am

Yes. More characters required

Hooverdan · May 19, 2023, 2:42pm

Then I am unfortunately out of ideas. Once @phillxnet is back he possibly has some better thoughts on this.

jihiggs · May 19, 2023, 5:54pm

It’s not something I need resolved. I’m fine managing permissions manually. But it does seem to be a bug when importing btrfs pools that existed in a prior install. And that the error message produces no message is probably something that should be looked at.

phillxnet · May 21, 2023, 2:30pm

@Hooverdan Thanks for your excellent run-down on our shares owner.group mechanisms earlier in this thread. This would make for a nice doc extension actually.

@jihiggs Hello again.
Re:

Agreed and we are aware of this. But as indicated by @Hooverdan we have it referenced in the docs only in our CentOS which is tangential really as back in that day/days there was a default group named identically to each users. But in OpenSUSE the default group is users. But there is still the underlying import bug that you have observed. It’s actually more of a bug of omission really, We don’t as @Hooverdan mentioned use a proper source of truth (the system) and reference only the db; which is find for Web-UI only interaction (the vast majority of our intended use) and this could/should be built different but just isn’t yet. But more serious is an absense to set the db entries at least on import.

So in short we need a GitHub issue to establish these db entries on ownership and access (while we still work that way) at least during an import. If you fancy creating such an issue that would be good as you are the one of the first to have raised this issue. As you say there are work arounds, but it catches folks out at the worst time.

The caveat to this proposed “enhance share import re ownership & access” issue is that we have a chicken and egg scenario. Normally folks will, and we advise to, import pools/shares then apply config backup. But the config backup will contain the new users etc. So something we have to watch for. Especially give user name and number restore, and the same for group. But again - now we have group attention on this issue it may be a good time to round out exactly what we do during import re the ownership assertion. We’d need to take a fresh look at our user/group import code here:

github.com

rockstor/rockstor-core/blob/bdc3df2762ceddf355b482ca50f698654e342861/src/rockstor/storageadmin/views/config_backup.py#L77-L100


      
          def restore_users_groups(ml):
              logger.info("Started restoring users and groups.")
              users = []
              groups = []
              # Dictionary to map group pk to group name. Used to re-establishes user
              # to group name relationship.
              groupname_from_pk = {}
              for m in ml:
                  if m["model"] == "storageadmin.user":
                      users.append(m["fields"])
                  if m["model"] == "storageadmin.group":
                      groupname_from_pk[m["pk"]] = m["fields"]["groupname"]
                      groups.append(m["fields"])
          
          
    # order is important, first create all the groups and then users.
              for g in groups:
                  generic_post("{}/groups".format(BASE_URL), g)
              for u in users:
                  # Replace user record 'group' field pk value with resolved group name.
                  u["group"] = groupname_from_pk[u["group"]]

This file has been truncated. show original

We have to have the pools/shares imported first as they need to exist before exports (in config restore) can be attached to them. Hence we do have a little complexity here.

So do do ahead and create an issue for this import ommision so that we have attribution and a little sketch of how you think we can approach the restoring of users rights that don’t yet existing would also be good. I’m not, off the top of my head, sure how we do this all really. But it may be we need some kind of re-think on our ordering here: assuming we have a config backup in play. We definitely need to be able to import poolls without this config save file but the question of what we do with unknown users is a little tricky. But doable. Again I’ve not focuesed on this issue just yet but we just need a little attention in this area to get a large gain in usability but as with many things it’s prioritising and limited resource so do by all means initiate a new import extension as it may turn out to be no big deal which would be great.

Re:

There may still be other immutable issues here. There is no known reason, other than a ro share, that share access edits are not being applied. We havn’t had any other reports of this lately so it may be an unrelated issue or another immutable flag is in play somehow.

Thanks for your diligent reporting here by the way - always appreciated. And keep them coming, but do appreciate that we are a small development team currently.