Vertical Backup is provided as a single executable file. It is recommended to download it to a datastore directory:
mkdir /vmfs/volumes/datastore1/verticalbackup
cd /vmfs/volumes/datastore1/verticalbackup
wget http://verticalbackup.com/esxi/vertical
chmod u+x vertical
sha256sum vertical # e6d55b2999f6994cb366870618cf66914683f908b2f175c991ef38f0a54e88a1
Once we have the executable ready, we first need to choose a host id, and the storage to back up to:
vertical init esxi55 sftp://esxi@192.168.1.100/storage
Here we use esxi55 as the host id. Each ESXi host should have a unique host id. Backups will be uploaded to 192.168.1.100 via SFTP with the username being esxi and the storage directory will be storage (relative to the home directory). Multiple ESXi hosts can back up to the same storage, and in fact this is the recommended way as this enables cross-host deduplication.
Now we are ready to run the backup command. This is to back up the virtual machine named vm-ubuntu:
vertical backup vm-ubuntu
Vertical Backup assigns increasing revision numbers to backups, starting at 1 for the first backup. We can list previous backups using the list command:
vertical list
The restore command restores a virtual machine to a previous revision:
vertical restore vm-ubuntu -r 1
This command sets the virtual machine vm-ubuntu to be backed up 10 minutes after every hour:
vertical cron 10 vm-ubuntu
We can also provide the time in the hh:mm format to schedule a daily backup job.
vertical cron 11:30 vm-ubuntu
To set up email notifications, first run the email command:
vertical email smtp.gmail.com 587 username password from@gmail.com to@gmail.com
Then when we can run the backup command with the --email option. An email will be send to to@gmail.com after the backup is finished.
vertical backup vm-ubuntu --email
The --email option can also be provided to the cron command to enable email notifications for a scheduled backup job
vertical cron 0:30 vm-ubuntu --email
Below are all the commands and global options supported by Vertical Backup:
usage: vertical [-h] [-v] [--log] [--disable-script] [--print-stack]
{restore,prune,license,list,email,cron,init,backup,check,help}
...
commands:
{restore,prune,license,list,email,cron,init,backup,check,help}
init initialize the backup storage
backup back up virtual machines to the storage
restore restore virtual machine from previous backup
list list backups
check check the integrity of backups
prune remove specified backups
cron set up a cron job
email set up email notifications
license manage the license
benchmark run performance benchmarks
help display help information
optional arguments:
-h, --help show this help message and exit
-v increase output verbosity
--log enable log-style output
--disable-script don't run scripts before and after the command
--print-stack print the stack trace on error
usage: vertical init [-h] [-c SIZE] [-e] [--temp-dir DIR] host_id storage_url positional arguments: host_id a unique id to identify the host machine storage_url the url of the storage to back up to optional arguments: -h, --help show this help message and exit -c SIZE, --chunk-size SIZE the size of chunks (default is 1M) -e, --encrypt encrypt to the storage with a master password --temp-dir DIR directory for storing temporary files
The init command assigns the given host id that will be used by the backup command later. It also tries to connect to the storage backend specified by the storage url. If the storage backend has not been initialized before, it will initialize the storage backend by uploading a configuration file containing the chunk size as specified and other parameters. If the configuration file already exists it will download the file and use the existing configuration parameters instead, ignoring the chunk size provided.
The chunk size parameter indicates the size of chunks that Vertical Backup will split the disk files into. It is recommended to be a multiple of 1024, and the default size is 1 megabytes.
The -e option encrypts the storage backend. If the storage backend has not been initialized, then a storage password must be entered. If the storage has been initialized with this option before, then this option becomes required.
Currently there are 5 types of storage urls:
In the SFTP url, the storage path is relative to the user's home directory. An extra slash is needed between the server and the storage path for an absolute path.
Not that if an SFTP storage is used, you must configure the ESXi host to enable outgoing SSH connections.
Depending on the storage backend you specify, you may be prompted to enter passwords/keys required to access the specified storage backend. To avoid entering them every time the storage is accessed, you can save them in environment variables or a special file. Please refer to the Managing Credential sections for details.
Under the hood, this command creates a subdirectory named .verticalbackup under the directory where the Vertical Backup executable is. It then saves the host id and the storage url to a file named preference under .verticalbackup. Later on other commands always read from this file to get the host id and the storage url, among other things.
The .verticalbackup directory is also the place for storing other auxiliary files. For instance, Vertical Backups caches chunks referenced by backup files and these chunks are temporarily stored in .verticalbackup/cache . You can use the --temp-dir option to change location of this cache directory.
This command also creates a custom firewall rule that enables outgoing HTTPS connections via port 443. This is needed since Vertical Backup needs to contact our license server at https://vertical-portal.appspot.com in order to activate licenses.
usage: vertical backup [-h] [-p] [--no-quiesce] [-t TAG] [--threads THREADS]
[-l LIMIT_RATE] [-e EXCLUDES] [--email]
[--subject EMAIL_SUBJECT]
[--failure-subject FAILURE_SUBJECT] [--parallel]
[--exclude-disk EXCLUDED_DISKS]
[vmname [vmname ...]]
positional arguments:
vmname the name of the virtual machine to back up
optional arguments:
-h, --help show this help message and exit
-p, --preserve-snapshots don't remove or create snapshots
--no-quiesce do not quiesce the virtual machine before the backup
-t TAG, --tag TAG specify a backup tag
--threads THREADS number of uploading threads
-l RATE, --limit-rate RATE limit the upload speed (KiloBytes/sec)
-e VMNAME, --exclude VMNAME do not backup the virtual machine with this name
--email email log after backup is done
--subject EMAIL_SUBJECT the subject of the email to be sent
--failure-subject SUBJECT the email subject to use when backup fails
--parallel allow multiple backups at the same time
--exclude-disk DISKNAME do not backup the disk with this name
--no-chunk-listing do not list chunks for initial backup
--snapshot-timeout TIMEOUT snasphot operation timeout (in seconds)
--power-off power off the virtual machine before backup
--power-on power on the virtual machine after backup
The backup command creates backups for one or more that one virtual machine, and uploads them to the storage backend specified by the init command.
By default, for each virtual machine to be backed up, all existing snapshots are removed at first and then a new one is created. This is to ensure that disk files are in a most recent state. After the backup is finished, the newly created snapshot will be removed. The -p option disables all these snapshot operations.
If --no-quiesce option is specified, the latest snapshot will be created with the Quiesce flag.
You can assign a tag to the backup to be created by using the -t option. In other commands you can select backups by their tags.
The backup operation supports multithreading if a number greater than 1 is given to the --threads option, or rate limiting if a non-zero number is given to the -l option.
The -e option excludes virtual machines from the list to be backed up. Wildcard characters such as * and ? can be used in both the -e optin and the vmname argument.
To send out an email after the backup is done, specify the --email option and optionally the --subject option to use a customized subject. The subject can be changed by the --failure-subject option if any error occurs during the backup. The email command must be invoked first to set up email configuration.
By default, Vertical Backup will not start if there is another process that has been running. The --parallell option will ignore the previous instance
The --exclude-disk option is used to exclude disks with certain names. Note that the virtual machine restored from a backup created with this option may not work properly due to the missing disks.
The --no-chunk-listing option can be used to avoid listing all chunks on the storage when running the first backup for a virtual machine. By default, when a virtual machine is backed up for the first time, Vertical Backup will construct a chunk cache in memory from the complete chunk list on the storage. The advantage of doing so is that if a chunk already in the storage, a lookup API call can be avoided. However, if there are too many chunks in the storage this chunk cache may consume a lot of memory. This option effectively disable the chunk cache at the cost of one lookup API call for each chunk to be uploaded or skipped.
The --snapshot-timeout option set the maximum time for the snapshot creation operation. The default is 120 seconds.
usage: vertical restore [-h] -r REVISION [-f VM] [--threads THREADS] [-l RATE] vmname [file [file ...]] positional arguments: vmname the name of the virtual machine to restore file only restore specified files required arguments: -r REVISION the revision of the backup to restore from optional arguments: -h, --help show this help message and exit -f VM, --restore-from VM restore from a backup of another virtual machine --power-off power off the virtual machine before restore --power-on power on the virtual machine after restore --threads THREADS number of downloading threads -l RATE, --limit-rate RATE limit the download speed (kilobytes/sec)
The restore command restores a given virtual machine to a previous revision. By default the virtual machine must be stopped before running this command, unless the --power-off option is specified
If the name of the virtual machine is the single argument, then it will restore all files contained in the backup. You can pass the name of files you wish to restore after the first argument, then these files will be the only files to be restored.
The first argument can also be a name of a directory. In this case the files to be restored will be written to this directory. The --restore-from option becomes required, which specifies the name of the virtual machine to be restored from. To restore from a virtual machine running on a different host but backing up to the same storage url, specify the virtual machine name in the format of vmname@hostid.
The restore operation supports multithreading if a number greater than 1 is given to the --threads option, or rate limiting if a non-zero number is given to the -l option.
The --power-off option can be used to power off the virtual machine before the restore operation, and the --power-on option to power on the virtual machine after the restore
operation.
usage: vertical list [-h] [-a] [-f] [vmname [vmname ...]] positional arguments: vmname list previous backups of the specified virtual machines optional arguments: -h, --help show this help message and exit -a, --all list all backups in the storage -f, --files show file information for each backup
The list command lists existing backups. By default it only lists backups created locally, but the -a option will force it to list all backups present on the same storage, regardless of their host ids.
Virtual machines can also be specified in the format of vmname@hostid, and wildcard patterns are allowed. Therefore, *@host1 will check all virtual machines on the host with host id host1, and *@* is equivalent to the -a option.
The -f option will also list detailed information about each file.
usage: vertical check [-h] [-a] [-r REVISIONS] [-t TAG] [-v] [vmname [vmname ...]] positional arguments: vmname verify backups of the specified virtual machines optional arguments: -h, --help show this help message and exit -a, --all check all virtual machines -r REVISIONS the revisions to verify -t TAG, --tag TAG only verify backups with the specified tag -v, --verify verify integrity of all chunks
The check command verifies that all chunks referenced by backups of the specified virtual machines exist.
If the -a option is specified, it will check backups of every virtual machine in the storage (including those in different hosts). Otherwise, it will only check specified virtual machines.
Virtual machines can also be specified in the format of vmname@hostid, and wildcard patterns are allowed. Therefore, *@host1 will check all virtual machines on the host with host id host1, and *@* is equivalent to the -a option.
The -r options, which can occur multiple times, limit the check to backups with the specified revisions numbers. Ranges of revision numbers can also be specified in the format of a-b, where a is the starting revision number and b is the ending revision number.
If the -t option is specified, the check will be further limited to backups with the specified tag.
The -v option will not only check the existence of chunks, but also make sure that each chunk is not corrupted. Backups that pass the -v check are guaranteed to be complete. Note that this option makes the check operation much slower (due to the need to download every chunk).
usage: vertical prune [-h] [-k POLICY] [-r REVISIONS] [-t TAGS] [-d]
[--exhaustive] [--exclusive] [--collect-only] [--delete-only]
[vmname [vmname ...]]
positional arguments:
vmname delete backups of the specified virtual machines
optional arguments:
-h, --help show this help message and exit
-k POLICY, --keep POLICY specify the retention policy
-r REVISIONS delete backups with the specified revision numbers
-t TAGS, --tag TAGS delete backups with the specified tags
-d, --dry-run don't actually remove
--exhaustive find all unreferenced chunks
--exclusive assume no backup is in progress
--collect-only skip the fossil deletion step
--delete-only skip the fossil collection step
The prune command deletes backups of specified virtual machines from the storage. Unless the --exclusive option is specified, when a backup is removed from the storage by this command, all chunks it references will not immediately be removed and instead remain in the storage. However they cannot be accessible by new backups created after the prune command is finished. Next time the prune command is run, it will check if certain conditions are met. If they are met, those unreferenced chunks will be permanently removed. This is the central idea of Two-Step Fossil Collection algorithm that enables Lock-Free Deduplication.
Virtual machines can also be specified in the format of vmname@hostid, and wildcard patterns are allowed. Therefore, *@host1 will check all virtual machines on the host with host id host1.
If no virtual machines are specified, then the 3 backup selection options, -k, -r, and -t, apply to all virtual machines in the storage.
The retention policies are specified by the -k option, which accepts an argument in the format of two numbers n:m, where n indicates the number of days between two consecutive backups to keep, and m indicates that the policy only applies to backups at least m day old. If n is zero, any backups older than m days will be removed.
Here are a few sample retention policies:
vertical prune -k 1:7 # Keep 1 backup per day for backups older than 7 days vertical prune -k 7:30 # Keep 1 backup every 7 days for backups older than 30 days vertical prune -k 30:180 # Keep 1 backup every 30 days for backups older than 180 days vertical prune -k 0:360 # Keep no backup older than 360 days
Multiple -k options must be sorted by their m values in decreasing order. For instance, to combine the above policies into one line, it would become:
vertical prune -k 0:360 -k 30:180 -k 7:30 -k 1:7
The -r options, which can occur multiple times, limit the pruning to backups with the specified revisions numbers. Ranges of revision numbers can also be specified in the format of a-b, where a is the starting revision number and b is the ending revision number.
If the -t option is specified, the pruning will be further limited to backups with the specified tag.
The --exhaustive option will scan the list of all chunks in the storage, therefore it will find not only unreferenced chunks from deleted backups, but also chunks that become unreferenced for other reasons (for instance those caused by incomplete backups). It will also find any file that does not look like a chunk file at all. In contrast, a default prune command will only identify chunks referenced by backups to be deleted, but not by others.
The --exclusive option will assume that no other ESXi hosts are accessing the same storage, effectively disabling the Two-Step Fossil Collection algorithm. With this option, the prune command will immediately remove unreferenced chunks.
The -d option is used to test what changes the prune command would have done. It is guaranteed not to make any changes on the storage. The following command checks if the chunk directory is clean (i.e., if there are any unreferenced chunks, temporary files, or anything else):
vertical prune -d --exclusive --exhaustive # prints unreferenced chunks
The --delete-only option will skip the fossil collection step, while the --collect-only option will skip the fossil deletion step.
If there are multiple ESXi hosts connected to the same storage (which is in fact the suggested use case), it is recommended that only one host will perform periodic prune operations. Although the prune operation is not known to be resource consuming, it is even better if the prune operation is performed on a non-production ESXi host, or even by Duplicacy on a non-ESXi computer.
usage: vertical cron [-h] time ... positional arguments: time the scheduled time to run the job ... virtual machines to back up followed by options optional arguments: -h, --help show this help message and exit
The cron command schedules a backup job to be run at the specified time.
If the time is a one digit or two digit number, then it is scheduling an hourly backup job and the time is the number of minutes after the hour.
If the time is in the format of hour:minute, then it is scheduling a daily backup job to be run at the specified time.
For more flexibility, you can also specify the time in the standard 5 field crontab format.
If the time is the string disable, then it will disable the current job that has been scheduled.
You can specify any valid options of the backup command after the names of virtual machines to be backed up. For instance, this command schedules an hourly job to backup a virtual machine named vm1 with email notifications enabled:
vertical cron 0 vm1 --email --subject '"Backup for vm1 has been completed successfully"'
usage: vertical email [-h] [--tls] [--test]
server port username password from_address to_address
positional arguments:
server SMTP server hostname or ip
port SMTP port (normally 25, 465, or 587)
username username on the SMTP server
password password to the SMTP server
from_address sender address
to_address receiver address
optional arguments:
-h, --help show this help message and exit
--tls enable SSL/TLS when sending emails
--test send a test email
The email command sets up email configuration for the backup command. Note that this command doesn't send out any email, unless the --test option is given.
The --tls option indicates that the TLS protocol will be used when talking to the email server.
The --test option will instruct the command to send out a test email after saving the configuration.
This command also creates a custom firewall rule that enables outgoing SMTP or SMTPS connections via the specified port.
usage: vertical benchmark [-h] [-l] [datastore] positional arguments: datastore specify the directory for testing disk access speeds optional arguments: -h, --help show this help message and exit -l, --long run longer tests
The benchmark command runs a few tests to measure the write and read speeds of the datastore, as well as the download and upload speeds of the storage.
By default this command selects the first datastore as the test directory, which can be changed by providing the optional datastore argument.
The -l option doubles the test time which may produce more accurate results.
usage: vertical license [-h] {activate,show} ...
positional arguments:
{activate,show}
show display license information
activate activate a new license
optional arguments:
-h, --help show this help message and exit
The license command supports 2 sub-commands.
The license show sub-command lists license information.
The license activate takes a license activation code, downloads the assigned license from our license server (at https://vertical-port.appspot.com), and then save the license which will be checked by the backup command.
Storage backends supported by Vertical Backup require various kinds of credentials, such as passwords, account ids, or keys. If you choose to encrypt the storage backend, then there is also the storage password.
Before prompting for user input, Vertical Backup will first check if the corresponding environment variables exist. If they do, Vertical Backup will read the credentials from these environment variables instead of keyboard input. If they don't, Vertical Backup will also check a file named passwords under the directory .verticalbackup (which is under the same directory as the Vertical Backup executable), where passwords can be stored as key-value pairs in json format.
The following table summarizes the name of the environment variable or the key in the passwords file for each type of credential:
| Credential | Environment variable | Key in .verticalbackup/passwords |
|---|---|---|
| storage password | VB_PASS | pass |
| SFTP password | VB_SSH_PASS | ssh_pass |
| SFTP key file | VB_SSH_KEY | ssh_key |
| SFTP key passphrase | VB_SSH_KEY_PASS | ssh_key_pass |
| Amazon S3 access id | VB_S3_ID | s3_id |
| Amazon S3 secret key | VB_S3_KEY | s3_key |
| Microsoft Azure account | VB_AZURE_ACCOUNT | azure_account |
| Microsoft Azure key | VB_AZURE_KEY | azure_key |
| Backblaze B2 account id | VB_B2_ACCOUNT | b2_account |
| Backblaze B2 application key | VB_B2_KEY | b2_key |
For instance, the following is a sample passwords file:
[root@esxi55:/vmfs/volumes/datastore1/vertical] cat .verticalbackup/passwords
{
"pass": "storage password",
"ssh_pass": "ssh password"
}
You can instruct Vertical Backup to run a script before or after executing a command. For example, if you create a script with the name pre-prune under the .verticalbackup/scripts directory, this script will be executed before the prune operation starts. A script named post-prune will be executed at the end of the prune operation. This rule applies to backup, restore, list, check, and prune commands.