[SOLVED] Cant delete empty snapshot folders created by Snapshot scheduled tasks

[t3kka]

I’ve been playing with different automated ways of performing send/receive to allow for a more resilient backup setup and during this I noticed that in one of my shares I have what appears to be some orphaned directories that were created by the Scheduled task jobs I had created to perform snapshots on the shares.

So for example, I have the following:

[root@rockstor ]# tree -L 2 -a /mnt2/Docker-Persistent-Storage/                                                                                
/mnt2/Docker-Persistent-Storage/                                                                                                                      
├── .Docker-Persistent-Storage-Backup_201809260200                                                                                                    
├── .Docker-Persistent-Storage-Backup_20180927                                                                                                        
├── .Docker-Persistent-Storage-Backup_201809280300                                                                                                    
├── .Docker-Persistent-Storage-Backup_201809290300                                                                                                    
├── .Docker-Persistent-Storage-Backup_201809300300                                                                                                    
├── ombi                                                                                                                                              
│   └── config                                                                                                                                        
├── portainer                                                                                                                                         
│   ├── compose                                                                                                                                       
│   ├── config.json                                                                                                                                   
│   ├── data                                                                                                                                          
│   ├── portainer.db                                                                                                                                  
│   ├── portainer.key                                                                                                                                 
│   ├── portainer.pub                                                                                                                                 
│   └── tls

The ‘.Docker-Persistent-Storage-Backup*’ directories were created as part of a scheduled task config’ed via the GUI. During my testing I had decided to remove the scheduled task and also to delete the relevant snapshots listed in the Shares > Snapshots for the aforementioned Share. Also reviewed the overall ‘Snapshots’ page in the GUI to ensure no other snapshots existed.

They all were removed fine from the UI but when navigating around in the CLI they still appear. They are not sub volumes (btrfs sub del doesn’t work) and a simple rm -rf also doesn’t work (says “operation not permitted” even as root user). They dont appear to be impacting anything but my OCD certainly finds them troublesome. Also, I’d just like to understand why they cant be simply removed…

Thanks for any input.

[phillxnet (Philip Guyton)]

@t3kka Hello again, I can chip in on this one:

@t3kka

They dont appear to be impacting anything but my OCD certainly finds them troublesome. Also, I’d just like to understand why they cant be simply removed…

Yes, we have tracked this one down to our use of setting and un-setting the immutable flag on directories to avoid the scenario where a failure to mount a subvol doesn’t end up with it’s mount point being populated via a potential share export. We have more recently fixed, in the stable channel updates, a couple of instances where the code failed to un-set these flags to the detriment of normal functions but there may be some remaining.

See the following thread where @Haioken and @Rene_Castberg reported an earlier more problematic instance of this:

SOLVED: BTRFS Subvolume create/delete fail

Problem has been resolved, though I’m unsure why it had happened. Immutable flag was set on /mnt2/tempraid, meaning that the contents of /mnt2/tempraid was unmodifiable (though decendants could be altered). I know I didn’t set the flag, and according to BTRFS IRC, scrubs do not set the flag in the event of error. Problem was resolved with chattr -i /mnt2/tempraid Should have thought of it before - in fact, I could swear I checked it, but apparently I did not.

which has now been sorted, and a later report by @satello of a similar instance:

[solved] Unable to create new shares

Hi, hope you can help. Been playing with RS for about a week, so far so good. I’ve installed a plex server and all is running smoothly. now i’m trying to create a new share for transmission and it pops an error. recreate: storage > shares > create new here is the msg Error running a command. cmd = /sbin/btrfs subvolume create -i 2015/24 /mnt2/rockstor_rockstor/transm-data. rc = 1. stdout = [“Create subvolume ‘/mnt2/rockstor_rockstor/transm-data’”, ‘’]. stderr = [‘ERROR: cannot create subvolum…

which has also no be fixed in the stable channel via the pull request indicated in that thread.

Hope that helps to explain this ‘operation not permitted’ thing.

[t3kka]

Hey @phillxnet, thanks! I was doing a lot of sleuthing around after I posted and I stumbled on a thread somewhere that also mentioned something about checking the folder attributes and sure enough when I looked at the lsattr of the folders in question they had the ‘i’ flag set. After removing the flag I still had issue removing them because they for whatever reason were currently in use. I rebooted which released whatever was still holding on to them and I could successfully remove them.

Further digging around showed a similar issue had also occurred with some shares/NFS exports that I had deleted in the GUI but were still showing up in the filesystem (albeit empty). Those too were fixed by simply changing the immutable flag and doing a rm -r.

I’ll mark as solved and again, thanks for the quick response!!!