Built on openSUSE testing channel live (early-adopters/developers only)

Flox · November 28, 2020, 12:46am

Hello again, @lexxa

That unfortunately seems like an issue with Duplicati rather than Rockstor. I quickly lookep it up and it seems to have been a “somewhat common” issue with Duplicati. For instance, I found an issue on their Github page with similar behavior suspecting a memroy leak that seems to have been fixed by a Duplicati update:

github.com/duplicati/duplicati

Suspected memory leak

opened 01:03PM - 18 Feb 20 UTC

closed 10:18PM - 15 Aug 20 UTC

fred-cardoso

performance issue

- [x] I have searched open and closed issues for duplicates. ----------------…------------------------ ## Environment info - **Duplicati version**: 2.0.5.101 - **Operating system**: Linux - **Backend**: OneDrive v2 ## Description Hello guys, I wanted to message someone before I opened this ticket so I could point more directly to the supposed bug. So the issue is the following. I was doing a backup whenit failed with this error: `2020-02-18 12:51:18 +00 - [Error-Duplicati.Library.Main.Operation.TestHandler-FailedToProcessFile]: Failed to process file duplicati-....` A more detailed analysis returned the following information: `"Key": "Error", "Value": "Insufficient memory to continue the execution of the program."` At this point I went on I checked the RAM usage: ![image](https://user-images.githubusercontent.com/32043667/74738153-38acd900-524e-11ea-8fd8-bf5419379206.png) At the end of the graph, you can see that there are "two steps". This steps were two tries of backing up some data, where you can see that the memory keeps going up and so forth. This is not the first time I see this error, actually it's the 3rd. I know that is related to duplicati because it only happens when I run duplicati, where the memory usage keeps increasing. Also, after restarting duplicati docker instance, the free memory increases in more than 1GB. ![image](https://user-images.githubusercontent.com/32043667/74738515-fc2dad00-524e-11ea-93a3-10a71e5d27d3.png) How can I try to debug this even further to find the culprit? ## Debug log `{ "DeletedFiles": 0, "DeletedFolders": 0, "ModifiedFiles": 0, "ExaminedFiles": 640, "OpenedFiles": 0, "AddedFiles": 0, "SizeOfModifiedFiles": 0, "SizeOfAddedFiles": 0, "SizeOfExaminedFiles": 665451250, "SizeOfOpenedFiles": 0, "NotProcessedFiles": 0, "AddedFolders": 0, "TooLargeFiles": 0, "FilesWithError": 0, "ModifiedFolders": 0, "ModifiedSymlinks": 0, "AddedSymlinks": 0, "DeletedSymlinks": 0, "PartialBackup": false, "Dryrun": false, "MainOperation": "Backup", "CompactResults": null, "VacuumResults": null, "DeleteResults": { "DeletedSetsActualLength": 0, "DeletedSets": [], "Dryrun": false, "MainOperation": "Delete", "CompactResults": null, "ParsedResult": "Success", "Version": "2.0.5.101 (2.0.5.101_canary_2020-01-23)", "EndTime": "2020-02-18T12:49:54.451689Z", "BeginTime": "2020-02-18T12:49:54.41959Z", "Duration": "00:00:00.0320990", "MessagesActualLength": 0, "WarningsActualLength": 0, "ErrorsActualLength": 0, "Messages": null, "Warnings": null, "Errors": null, "BackendStatistics": { "RemoteCalls": 9, "BytesUploaded": 0, "BytesDownloaded": 69866, "FilesUploaded": 0, "FilesDownloaded": 2, "FilesDeleted": 0, "FoldersCreated": 0, "RetryAttempts": 4, "UnknownFileSize": 0, "UnknownFileCount": 0, "KnownFileCount": 10, "KnownFileSize": 237451890, "LastBackupDate": "2020-02-18T11:25:05+00:00", "BackupListCount": 2, "TotalQuotaSpace": 1104880336896, "FreeQuotaSpace": 369315255643, "AssignedQuotaSpace": -1, "ReportedQuotaError": false, "ReportedQuotaWarning": false, "MainOperation": "Backup", "ParsedResult": "Success", "Version": "2.0.5.101 (2.0.5.101_canary_2020-01-23)", "EndTime": "0001-01-01T00:00:00", "BeginTime": "2020-02-18T12:49:39.279659Z", "Duration": "00:00:00", "MessagesActualLength": 0, "WarningsActualLength": 0, "ErrorsActualLength": 0, "Messages": null, "Warnings": null, "Errors": null } }, "RepairResults": null, "TestResults": { "MainOperation": "Test", "VerificationsActualLength": 3, "Verifications": [ { "Key": "duplicati-20200218T112505Z.dlist.zip.aes", "Value": [] }, { "Key": "duplicati-i40347f0a0cb44c1f90da53a9f0fd2923.dindex.zip.aes", "Value": [] }, { "Key": "duplicati-b69425c1293cc4481993cedd83b87cbe4.dblock.zip.aes", "Value": [ { "Key": "Error", "Value": "Insufficient memory to continue the execution of the program." } ] } ], "ParsedResult": "Success", "Version": "2.0.5.101 (2.0.5.101_canary_2020-01-23)", "EndTime": "2020-02-18T12:51:18.731577Z", "BeginTime": "2020-02-18T12:49:55.950327Z", "Duration": "00:01:22.7812500", "MessagesActualLength": 0, "WarningsActualLength": 0, "ErrorsActualLength": 0, "Messages": null, "Warnings": null, "Errors": null, "BackendStatistics": { "RemoteCalls": 9, "BytesUploaded": 0, "BytesDownloaded": 69866, "FilesUploaded": 0, "FilesDownloaded": 2, "FilesDeleted": 0, "FoldersCreated": 0, "RetryAttempts": 4, "UnknownFileSize": 0, "UnknownFileCount": 0, "KnownFileCount": 10, "KnownFileSize": 237451890, "LastBackupDate": "2020-02-18T11:25:05+00:00", "BackupListCount": 2, "TotalQuotaSpace": 1104880336896, "FreeQuotaSpace": 369315255643, "AssignedQuotaSpace": -1, "ReportedQuotaError": false, "ReportedQuotaWarning": false, "MainOperation": "Backup", "ParsedResult": "Success", "Version": "2.0.5.101 (2.0.5.101_canary_2020-01-23)", "EndTime": "0001-01-01T00:00:00", "BeginTime": "2020-02-18T12:49:39.279659Z", "Duration": "00:00:00", "MessagesActualLength": 0, "WarningsActualLength": 0, "ErrorsActualLength": 0, "Messages": null, "Warnings": null, "Errors": null } }, "ParsedResult": "Error", "Version": "2.0.5.101 (2.0.5.101_canary_2020-01-23)", "EndTime": "2020-02-18T12:51:18.75452Z", "BeginTime": "2020-02-18T12:49:39.279649Z", "Duration": "00:01:39.4748710", "MessagesActualLength": 20, "WarningsActualLength": 0, "ErrorsActualLength": 1, "Messages": [ "2020-02-18 12:49:39 +00 - [Information-Duplicati.Library.Main.Controller-StartingOperation]: The operation Backup has started", "2020-02-18 12:49:39 +00 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Started: ()", "2020-02-18 12:49:50 +00 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Completed: (10 bytes)", "2020-02-18 12:49:54 +00 - [Information-Duplicati.Library.Main.Operation.DeleteHandler-DeleteResults]: No remote filesets were deleted", "2020-02-18 12:49:54 +00 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Started: ()", "2020-02-18 12:49:55 +00 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: List - Completed: (10 bytes)", "2020-02-18 12:49:55 +00 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-20200218T112505Z.dlist.zip.aes (64,78 KB)", "2020-02-18 12:50:01 +00 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-20200218T112505Z.dlist.zip.aes (64,78 KB)", "2020-02-18 12:50:01 +00 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-i40347f0a0cb44c1f90da53a9f0fd2923.dindex.zip.aes (3,45 KB)", "2020-02-18 12:50:02 +00 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Completed: duplicati-i40347f0a0cb44c1f90da53a9f0fd2923.dindex.zip.aes (3,45 KB)", "2020-02-18 12:50:02 +00 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-b69425c1293cc4481993cedd83b87cbe4.dblock.zip.aes (181,86 MB)", "2020-02-18 12:50:09 +00 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Retrying: duplicati-b69425c1293cc4481993cedd83b87cbe4.dblock.zip.aes (181,86 MB)", "2020-02-18 12:50:19 +00 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-b69425c1293cc4481993cedd83b87cbe4.dblock.zip.aes (181,86 MB)", "2020-02-18 12:50:27 +00 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Retrying: duplicati-b69425c1293cc4481993cedd83b87cbe4.dblock.zip.aes (181,86 MB)", "2020-02-18 12:50:37 +00 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-b69425c1293cc4481993cedd83b87cbe4.dblock.zip.aes (181,86 MB)", "2020-02-18 12:50:47 +00 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Retrying: duplicati-b69425c1293cc4481993cedd83b87cbe4.dblock.zip.aes (181,86 MB)", "2020-02-18 12:50:57 +00 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-b69425c1293cc4481993cedd83b87cbe4.dblock.zip.aes (181,86 MB)", "2020-02-18 12:51:04 +00 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Retrying: duplicati-b69425c1293cc4481993cedd83b87cbe4.dblock.zip.aes (181,86 MB)", "2020-02-18 12:51:14 +00 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Started: duplicati-b69425c1293cc4481993cedd83b87cbe4.dblock.zip.aes (181,86 MB)", "2020-02-18 12:51:18 +00 - [Information-Duplicati.Library.Main.BasicResults-BackendEvent]: Backend event: Get - Failed: duplicati-b69425c1293cc4481993cedd83b87cbe4.dblock.zip.aes (181,86 MB)" ], "Warnings": [], "Errors": [ "2020-02-18 12:51:18 +00 - [Error-Duplicati.Library.Main.Operation.TestHandler-FailedToProcessFile]: Failed to process file duplicati-b69425c1293cc4481993cedd83b87cbe4.dblock.zip.aes" ], "BackendStatistics": { "RemoteCalls": 9, "BytesUploaded": 0, "BytesDownloaded": 69866, "FilesUploaded": 0, "FilesDownloaded": 2, "FilesDeleted": 0, "FoldersCreated": 0, "RetryAttempts": 4, "UnknownFileSize": 0, "UnknownFileCount": 0, "KnownFileCount": 10, "KnownFileSize": 237451890, "LastBackupDate": "2020-02-18T11:25:05+00:00", "BackupListCount": 2, "TotalQuotaSpace": 1104880336896, "FreeQuotaSpace": 369315255643, "AssignedQuotaSpace": -1, "ReportedQuotaError": false, "ReportedQuotaWarning": false, "MainOperation": "Backup", "ParsedResult": "Success", "Version": "2.0.5.101 (2.0.5.101_canary_2020-01-23)", "EndTime": "0001-01-01T00:00:00", "BeginTime": "2020-02-18T12:49:39.279659Z", "Duration": "00:00:00", "MessagesActualLength": 0, "WarningsActualLength": 0, "ErrorsActualLength": 0, "Messages": null, "Warnings": null, "Errors": null } }`

I would refer you to the Duplicati project for help about that, unfortunately, as they would be the best positioned to help. The only thing I would suggest is to make sure you are running the latest docker image.

I’m sorry I couldn’t be of more help here.