Multi period snapshot schedule not working correct

I have created a multi period snapshot schedule for my shares. In previous versions of Rockstor this worked flawless. But after a recent reinstall a few months ago it seems this isn’t working properly anymore.

My schedule is:
A daily snapshot is created, with a maximum of 7 snapshots. A weekly snapshot is created, with a maximum of 5 snapshots. A monthly snapshot is created, with a maximum of 12 snapshots. A yearly snapshot is created, with a maximum of 5 snapshots.

I can see that the snapshots are created, but for each share a maximum of 7 snapshots is available. These are all recent, created by the daily schedule. The older snapshots created by the weekly or monthly schedule seem to be deleted by the daily schedule (because it sees more than 7 snapshots).
In this screenshot you can see that for my photo share 7 snapshots are available: 6 from the daily snapshot task and 1 from the weekly snapshot task. The previous snapshots form the monthly and weekly tasks seem to have been deleted.
rockstor%20snapshots

As you can see in my schedule, a monthly snapshot has been created 22 days ago. However it is not visible in the snapshots of the share (and neither is the one from the previous month).
rockstor%20snapshots%202

Can anyone see what I am doing wrong? Or has something changed in how the snapshot task schedule work?

2 Likes

Hi @sanderweel,

Nice catch, there! I personally haven’t worked on that part of Rockstor yet, but let’s see if I can help before somebody more knowledgeable comes in. I could thus be completely wrong, but I might have an idea in mind so I’ll try to describe what I can tell from the process so that you can see if what I think makes sense.

First, it all seems fine from your screenshots, but we do need to make sure all scheduled snapshots are indeed registered as they should. For these scheduled tasks, Rockstor simply uses crontab, so we can first make sure they are written as they should in there; this can easily be done as follows:

grep st-snapshot /etc/cron.d/rockstortab

You should see an entry for all the scheduled tasks of type “snapshot” you have setup in the web-UI. As a result, you will see a lot more than those for your “Fotos” share. I don’t have access to my Rockstor systems at this very moment, but you should see lines ending with (...) /opt/rockstor/bin/st-snapshot <id>. You should thus have as many of these lines as scheduled tasks of type “snapshot”.

You can thus see that this crontab will call the following script:

The part of interest in our case is below:


This is the bit that gets all the related snapshots, orders them, and deletes all those that are > max_count (maximum number of snapshots to keep set in the web-UI).
What I think could be the source of your problem here is in the following call:

    snapshots = Snapshot.objects.filter(
        share=share, snap_type=snap_type, name__startswith=prefix
    ).order_by("-id")

In plain english:

  • Snapshot.objects.filter: we get all the snapshot that fit the criteria below
  • share=share: share in question
  • snap_type=snap_type: snapshots set by the task scheduler
  • name__startswith=prefix: snapshots that have the prefix set in the web-UI

Now here, I’m curious about the last one. Indeed, I can see a problem in case one uses the same prefix for different scheduled snapshot tasks of the same share as the Snapshot.objects.filter() above will pick all those entries that have the same prefix.
Thanks to your screenshot (thanks again, those are always very useful!), it seems you have used the prefix SNAPSHOT for all the scheduled snapshot tasks for your Fotos share, am I understanding this correctly? If yes, that might explain why your older snapshots were deleted even though they weren’t supposed to.

I would thus first verify that the crontab is written properly (to my knowledge, we haven’t had reports of this lately but we never know), and then I would check what prefix you used for your snapshots of this share. If my hunch is correct, using different prefixes would be a fix for your issue. This would definitely need to be improved, of course, but let’s make sure this is actually what is happening here first and then we can think of ways to improve that.

Hope this helps,

2 Likes

I have 27 scheduled snapshots, the result of ‘grep st-snapshot /etc/cron.d/rockstortab’ is:
0 0 * * * root /opt/rockstor/bin/st-snapshot 35 *-----*
0 1 * * * root /opt/rockstor/bin/st-snapshot 36 *-----*
30 0 * * * root /opt/rockstor/bin/st-snapshot 37 *-----*
30 2 * * 4 root /opt/rockstor/bin/st-snapshot 38 *-----*
30 2 * * 2 root /opt/rockstor/bin/st-snapshot 40 *-----*
30 2 * * 3 root /opt/rockstor/bin/st-snapshot 41 *-----*
30 2 * * 1 root /opt/rockstor/bin/st-snapshot 42 *-----*
30 2 * * 0 root /opt/rockstor/bin/st-snapshot 43 *-----*
30 2 * * 6 root /opt/rockstor/bin/st-snapshot 44 *-----*
30 2 * * 5 root /opt/rockstor/bin/st-snapshot 45 *-----*
0 3 * * 4 root /opt/rockstor/bin/st-snapshot 46 *-----*
0 4 2 * * root /opt/rockstor/bin/st-snapshot 47 *-----*
30 4 2 * * root /opt/rockstor/bin/st-snapshot 48 *-----*
0 5 2 * * root /opt/rockstor/bin/st-snapshot 49 *-----*
30 5 2 * * root /opt/rockstor/bin/st-snapshot 50 *-----*
0 4 3 * * root /opt/rockstor/bin/st-snapshot 51 *-----*
30 4 3 * * root /opt/rockstor/bin/st-snapshot 52 *-----*
0 5 3 * * root /opt/rockstor/bin/st-snapshot 53 *-----*
30 5 3 * * root /opt/rockstor/bin/st-snapshot 54 *-----*
0 0 5 1 * root /opt/rockstor/bin/st-snapshot 55 *-----*
30 0 5 1 * root /opt/rockstor/bin/st-snapshot 56 *-----*
0 1 5 1 * root /opt/rockstor/bin/st-snapshot 57 *-----*
30 1 5 1 * root /opt/rockstor/bin/st-snapshot 58 *-----*
0 2 5 1 * root /opt/rockstor/bin/st-snapshot 59 *-----*
30 2 5 1 * root /opt/rockstor/bin/st-snapshot 60 *-----*
0 3 5 1 * root /opt/rockstor/bin/st-snapshot 61 *-----*
30 3 5 1 * root /opt/rockstor/bin/st-snapshot 62 *-----*

So that’s 27 lines, seems crontab is written properly.

I have used the same prefix for all snapshots, because I want to be able to use them in Windows as shadow copy.

1 Like

Thanks!

Oh good point… I would need to refresh myself on shadow copies to see what can be done here but the time being, it seems only one of the yearly/monthly/weekly/daily schedules can be used for it. This is definitely something that we need to improve, so would you be ok to open an issue on our Github repo linking back to this thread so that we can keep track of it?

No problem if you can’t, but let us know so that we can make sure it doesn’t get forgotten.

2 Likes

I have created a new github issue for this: https://github.com/rockstor/rockstor-core/issues/2323

Using different prefixes fixes the issue, but like already noted, you will have to choose which of the schedules will be available in Windows. Ideally, I would like to be able to use the multi period snapshots and have all of them available as Windows shadow copy.

2 Likes

I’m glad that indeed fixed it.

Thanks a lot for creating the issue and linking back to it. We’ll have to come up with a way to accommodate this but now that we have this issue opened, we’ll be able to follow up on it.

Thanks again!

1 Like