Monday, 17 September 2012

VMware ESXi fastcopy.py to quickly copy .vmdk files


Overview


fastcopy.py is a python script for VMware ESXi 4.0 - 5.x that can be used as a replacement for "cp -R" to copy Virtual Machines.
fastcopy use cp to copy normal files but use vmkfstools to copy .vmk files.
Because vmkfstools is 10 to 20 time faster to copy such files, you will see a big difference.
When copying .iso files you will not notice any difference !

The typical use of fastcopy.py is to copy multiple directories containing VMs to a target directory.
fastcopy.py don't handle recursive copy or simple file copy. Sources and target must be directories.

Usage

For example top copy one VM to a new directory :

./fastcopy.py  /vmfs/volumes/SATA1/windows2008 /vmfs/volumes/NEWSATA

or if you want to copy multiple VMs, just list all VM directories. The last one must be the target one.

./fastcopy.py  /vmfs/volumes/SATA1/windows2008 /vmfs/volumes/SATA1/linux /vmfs/volumes/NEWSATA


Installation

Cut&Paste script below to one of your datastore and don't  forget to make it executable using chmod :

chmod a+x

Here is fastcopy.py

#!/bin/python
# fastcopy.py
# copy VMware virtual machine directories using cp and vmkfstools
# (c) alain.spineux@gmail.com
# released under GPL
# usage: fastcopy.py vm_dir1 vm_dir2 ... target_dir
#
# ATTN: don't copy recursively

import sys, os, subprocess, re

def copy(srcdir, dstdir):
    skipped=0
    for filename in os.listdir(srcdir):
        fullfilename=os.path.join(srcdir, filename)
        if os.path.isdir(fullfilename):
            print fullfilename, 'SKIPPED'
            skipped+=1
            continue
        print filename
        if filename.endswith('-flat.vmdk'):
            # this is the data file, it will be copied by the .vmdk
            continue
        if re.match('.*-s[0-9]{3}.vmdk$', filename):
            # this is part of a sparse file, it will be copied by the .vmdk
            continue

        # dont use vmkfstools for snapshot files
        if filename.endswith('.vmdk') and \
            not re.match('.*-[0-9]{6}-delta.vmdk$', filename) and \
            not re.match('.*-[0-9]{6}.vmdk$', filename):
            args=['vmkfstools', '-i', fullfilename, os.path.join(dstdir, filename) ]
        else:
            args=['cp', fullfilename, os.path.join(dstdir, filename) ]
        # print args
        subprocess.call(args)
    return skipped

if len(sys.argv)<3:
    print 'Usage: fastcopy.py src_dir... target_dir'
    sys.exit(1)

srcdirs=map(lambda x:x.rstrip('/'), sys.argv[1:-1])
dstdir=sys.argv[-1].rstrip('/')

if not os.path.isdir(dstdir):
    print 'dst_dir must be a directories'
    sys.exit(1)

for srcdir in srcdirs:
    if not os.path.isdir(srcdir):
        print 'not a directory:', srcdir
        sys.exit(1)
    targetdir=os.path.join(dstdir, os.path.basename(srcdir))

    if os.path.exists(targetdir):
        print 'target dir already exists:', targetdir
        sys.exit(1)

skipped=0
for srcdir in srcdirs:
    targetdir=os.path.join(dstdir, os.path.basename(srcdir))
    os.mkdir(targetdir)
    skipped+=copy(srcdir, targetdir)

if skipped>0:
    print "SKIPPED:", skipped

3 comments:

  1. Thanks a lot for your script! I just did my migration in 30 min, previously it was taking hours...

    ReplyDelete
  2. Thanks heaps for this script too! It saved heaps of pain.
    We've just slightly modified it to be able to backup entire datastore with some smart logic around overwriting previously backup VMs.

    If you're interested in, you can get our version from here: http://pastebin.com/da7HxVug

    ReplyDelete
  3. Thanks for providing this informative information you may also refer.
    http://www.s4techno.com/vmware-training-pune-online/

    ReplyDelete