Morgan Jun 26 4:28PM 2017
I'm trying to find a suitable backup option for some VM's I am setting up under esxi. Ideally I'd like those to be backed up each to an Amazon EC2 instance, so that if the primary machine ever fails, I can swap over to a cloud machine.
I stumbled across Vertical Backup, and it looks like it may do this, but I am not clear on exactly how. I've reviewed the "guide" but that document doesn't really say anything about what happens at the destination end where backups are received.
Can you provide guidance as to the following: 1) How the restore process works 2) How the restore process would fit in with a failover scenario to a cloud VM?
Thank you, Morgan
gchen Jun 28 9:54AM 2017
Ok I'll explain how to restore a backup created on one host to another.
Suppose that you set up Vertical Backup on a host
host1 and create a backup for the virtual machine named
vertical init host1 s3://mybucket vertical backup myvm
Now on another
host2 you set up Vertical Backup with the same storage url but with a different host id:
vertical init host2 s3://mybucket
You can see the backup created by `host1' using the list command:
vertical list myvm@host1
To restore the backup on
host2, first create a directory in the datastore and then pass the directory to the restore command:
vertical restore \path\to\restore --restore-from myvm@host1 -r 1
You'll find all files related to the virtual machine
myvm have been restored under
\path\to\restore. You can now add that virtual machine to in vSphere client.
If later you create another backup for
vertical backup myvm
Then run this command on incrementally update the new vm on
vertical restore \path\to\restore --restore-from myvm@host1 -r 2
Did this answer your questions?
Morgan Jun 28 12:41PM 2017
Hi gchen, yes, that is helpful. However, it leaves me with several questions too:
1) I would like to back up VM's to a MacOS host that is running VMware Fusion. I have gotten the esxi machine connected and backing up to the Mac via SFTP. The question is, how do I restore on the MacOS machine? Do I use "duplicacy" on the Mac?
2) Your instructions imply that the recipient machine will also be esxi. Maybe I am missing something, but it seems like on Amazon EC2, one does not run esxi, instead, one imports a VM to their AMI format. So the backup would need restoring somehow within the virtual machine, not as a part of esxi..? I am relatively new to the VM-in-the-cloud world, so excuse me if I'm missing something about how I could make this work to have a "failover" VM running in the cloud.
Morgan Jun 28 1:07PM 2017
Okay, not a person to wait, I tried using duplicacy on the target mac - both command line and gui - to see if I can use it to restore from the vertical backup. I can't get it to work.
I had to create a "fake" repository to get either one to initialize. So I went ahead an initialized them, pointing them to the same data store being used for my vm backup.
However, I cannot figure out how to get either the command line or gui to recognize the already existing backup of the VMs created by vertical backup.
The question is this: How can I get a backup of a VM that's from an esxi host, restored onto a non-esxi machine?
gchen Jun 28 2:52PM 2017
You're almost there. If you create the fake repository using 'myvm@host1' as the repository id then you will be able to restore backups created for
duplicacy init myvm@host1 s3://mybucket duplicacy list duplicacy restore -r 1
I've never tried importing an ESXi vm to Amazon EC2. The problem with that in my opinion is that you can't convert the disk image incrementally. If you get a dedicated server capable of running ESXi from elsewhere like OVH or Hetzner, then it will be much faster to update an existing vm on your backup host (both Vertical Backup and Duplicacy support incremental restore).
Morgan Jun 28 6:21PM 2017
Thanks for the update - however, I can't get this to work.
When I do a command like
duplicacy init webuzo-lower-ed1@esximini1 /Volumes/BlueSky2/VM/backup/
"The repository /Volumes/TheBlueSky has already been initialized
(also note that it names the repository incorrectly as the entire volume, not just the specified sub directory).
Storage set to /Volumes/BlueSky2/VM/backup/
So maybe it is confused because I already initialized a local repository named VMs
Then I tried
duplicacy add VMs webuzo-lower-ed1@esximini1 /Volumes/BlueSky2/VM/backup/
The storage '/Volumes/BlueSky2/VM/backup/' has already been initialized Compression level: 100 Average chunk size: 1048576 Maximum chunk size: 1048576 Minimum chunk size: 1048576 Chunk seed: 766572746963616c6261636b7570 /Volumes/TheBlueSky will be backed up to /Volumes/BlueSky2/VM/backup/ with id webuzo-lower-ed1@esximini1
But there's nothing there!
Storage set to /Volumes/BlueSky2/VM/backup/
Not sure how to proceed?
I understand your point about VM backups to EC2. I do have a possible other option of a second server. However, I will also want to make the backups to my mac. I hope you can help make this easier.
gchen Jun 28 7:09PM 2017
repository is the directory to be backed up, and since you ran the init command under
/Volumes/TheBlueSky, that became the repository. The init command will create a hidden subdirectory
.duplicacy under the repository, and if you run the init command it will complain that the repository
/Volumes/TheBlueSky has been initialized. By removing the
.duplicacy subdirectory you will be able to initialize the repository again.
So in your case you should create a subdirectory under
/Volumes/TheBlueSky to be used as the repository:
cd /Volumes/TheBlueSky mkdir VMs cd VMs rm -rf .duplicacy duplicacy init webuzo-lower-ed1@esximini1 /Volumes/BlueSky2/VM/backup/ duplicacy list # should show you backups created from your ESXi
Morgan Jun 28 10:01PM 2017
Okay, thanks for helping me move forward.
I tried as you suggested. It seemed to proceed, then I got an error:
> duplicacy restore -r 1 Storage set to /Volumes/BlueSky2/VM/backup/ Restoring /Volumes/TheBlueSky/VMs to revision 1 Downloaded vmfs/volumes/datastore1/webuzo1/Webuzo.vmdk (526) Downloaded vmfs/volumes/datastore1/webuzo1/owncloudConv.vmdk (525) File vmfs/volumes/datastore1/webuzo1/owncloudConv-flat.vmdk has a mismatched hash: e7c5fca5119f94deb6e9635e72eb217a16bdd7da8d1eb6bd103c50ee80a7b1f1 instead of #ff5f259c0c589426f8c6e3f05ebc75c8a6e79afd9c81a26ae42b0a37749e8e80
I have no idea what went wrong... what shall I try next?
gchen Jun 28 11:23PM 2017
This is because Duplicacy and Vertical Backup use different ways to calculate the file hash, and the leading # means it is the one used by Vertical Backup. However, Duplicacy 2.0.0+ should correctly accept hashes created by Vertical Backup. Any chance you're running an old version?
Morgan Jun 29 10:52AM 2017
I just downloaded the duplicacy_osx_x64_2.0.3 from your releases page yesterday. That was the version I was using. I've never had duplicacy on any of my systems before yesterday.
gchen Jun 29 12:25PM 2017
Sorry my fault -- just realized that Duplicacy doesn't handle Vertical Backup-style hashes in the restore command. I'm working on a fix now.
gchen Jun 29 10:19PM 2017
If you download version 2.0.4 from our releases page on github.com it should be able to restore backups created by Vertical Backup.
Morgan Jul 1 8:12PM 2017
wow, thank you, that was fast. I will try it :)