MemError on prune

Suspense    Oct 17 9:12AM 2017

Hi,

Attempting to prune backups from a VM, upon doing so i get this error;

https://i.imgur.com/9nP4Qgv.png

Esxi host got over 20gig of ram available.

Regards


gchen    Oct 17 1:31PM 2017

Can you try this 1.1.4 build at http://acrosync.com/esxi/vertical (sha256: 370e4316d7f513608ff674be0b7c7080d625067b11efa6819c5debeb703e22fa)? I fixed a memory hogging bug where a list of referenced chunks should be cleared as soon as they have been all accounted for.


Suspense    Oct 19 2:23AM 2017

Hi!

So it seems to have worked, i think. Im not sure if i dont quite understand how this prune system works.

I run the prune command against revision 3 of my vm(got a total of 14 revisions), it tells me no previous fossil collections found. After a little while itll say "chunk xxxxx cannot be found in the storage"

Revision 3 still exists. So i run the command again and i get the same result.

This happens across all revisions i attempt to do this one, it seems nothing really gets deleted.

Can you explain how this works?

Regards


Suspense    Oct 19 3:29AM 2017

It seems i might have been inpatient, after half an our ish, it seems to have deleted a bunch fo chunks.

I think it works

Despite it saying it just removed revision 2, i still see revision 2 index file and its also shown when doing vertical list. Is this intended?

Regards


gchen    Oct 19 11:26AM 2017

When it reports the "chunk xxxxx cannot be found in the storage" massage, you'll need to run the check command with the -a switch.

Normally this message is caused by using the --exclusive option with the prune command when there are other backups in progress. There is a discussion about the same issue here: https://duplicacy.com/issue?id=5718639969828864. Although it was about Duplicacy, the prune command works the same way in Vertical Backup.

Also note that if you don't use the --exclusive option, the prune command will not remove unreferenced chunks immediately. Instead, it will only mark them as fossils and on next invocation if there are new backups it will attempt to remove fossils permanently. Please refer to https://github.com/gilbertchen/duplicacy/blob/master/DESIGN.md for more details.


Rob G    Jun 25 2:49PM 2018

I am having this issue with a fairly small set of backups (maybe 100 revisions, 30G or so of chunks) on a 128GB host, using 1.2.0:

Vertical Backup 1.2.0

Storage set to /vmfs/volumes/Backup

No previous fossil collections found

Traceback (most recent call last):

File "vertical_main.py", line 148, in

File "vertical_backup.py", line 781, in pruneCommand

File "vertical_snapshot.py", line 1507, in prune

File "vertical_snapshot.py", line 1517, in collectFossils

File "vertical_snapshot.py", line 332, in getSnapshotChunks

File "vertical_snapshot.py", line 308, in getChunkHashesFromSequence

File "vertical_snapshot.py", line 406, in downloadChunk

MemoryError

Failed to execute script vertical_main


gchen    Jun 25 10:12PM 2018

A process running directly inside an ESXi host can only use less than 1G memory, no matter how much memory the host has. Therefore, we would recommend using Duplicacy (running from a non-ESXi computer) to do the pruning. Using Duplicacy for this purpose is free even for commercial use.


Rob G    Jun 26 9:28AM 2018

I can do that. Do you have any recommendations on how much memory Duplicacy might need? (Obviously dependent on backup size, but baseline?)


gchen    Jun 26 3:28PM 2018

I don't have a good estimate but I guess in your case a few GB should be enough.


Log in to comment
Copyright © Acrosync LLC 2017