Carl Hjerpe Sep 8 3:28AM 2017
https://gist.github.com/Lillecarl/01af93ddba368e5fe64fb8504787da64
Idk if i should care about this or not? :)
gchen Sep 8 8:24AM 2017
I think you're using too many threads. Vertical Backup always uses one thread to scan the vmdk files, so having more than 4-8 threads normally doesn't help.
Carl Hjerpe Sep 9 5:43PM 2017
When i used rclone to transfer files i found that increasing the number of parralel file transfers increased speeds (only reached about 10Mbps on a single file transfer), which is why i spawn so many threads.
Doing tests with vertical backup shows that when using 50 threads i push 800 MB/s (I think it should be Mb, but that's what vertical tells me), now when using 8 threads as you suggest i'm only doing 97 MB/s.
Correct me if I'm wrong, but that's from personal experience :)
Best Regards Carl
gchen Sep 9 10:48PM 2017
Run ps | grep vertical to see if there are any vertical processes from previous runs. If no, run ulimit -p 256 to see if that helps.
800 MB/s may sounds unrealistic, but there are several factors that may contribute to it, such as compression, deduplication (chunks may already exist in the storage), sectors that are all zeros.
Carl Hjerpe Sep 12 7:10AM 2017
I'm not seeing 800 anymore, but as long as it bottlenecks because of hardware (CPU, IO, WAN) I'm very happy! :)
Here's a log from my last try with many threads :) (Earlier i used /opt/verticalbackup, but after reboot the folder was cleaned) https://gist.github.com/Lillecarl/b1f70fbe9971efa5e210b2c3be4b715e
gchen Sep 12 8:55AM 2017
Failed to run command '/bin/vim-cmd vmsvc/snapshot.removeall 6': No space left on device
If you manually run /bin/vim-cmd vmsvc/snapshot.removeall 6 from the shell does it give you the same error.
If so can you run vdf -h to see if it was one of the ram disks that ran out of space.