Basically a very simple setup, but have eluded me for some time now: Simply backup some running VM's on a ESXi 5.x to a FreeNAS NFS service. I'm running FreeNAS 8.2 (FreeNAS update routine is also give me some strange access rights errors lately, so never got round to updating the bugger) Shouldn’t be a problem with version 8.1 or 8.3.
Things might be done differently with better preference/lesser work. I'm no expert, so please let me know if your setup performs better, or to point out some stupid mistakes I have made. But the following will get backups of your VM's to a FreeNAS box.
I'm still struggling with some kind of synchronisation with Amazon S3 from FreeNAS. If you have done any kind of progress along offsite sync from FreeNAS or any throughs on this, please let me know.
All glory is going to William Lam (www.virtuallyghetto.com) for his work on ghettoVCB script. There is nothing original here, I just put some snips together.
1. What you need
2. NFS on FreeNAS
3. Set-up ESXi for the NFS store
4. Setting up ghettoVCB on the ESXi
5. Copy and extract the ghetto zip to your ESXi host.
6. Configure GhettoVCB
7. Finally running (or debugging) the script
1. What you need
This thing was tested on ESXi 5.0.0, 768111 and FreeNAS 8.2 (FreeNAS-8.2.0-RELEASE-p1-x64 (r11950). Either ESXi 4.0 or 5.1 shouldn’t be a problem with ghettoVCB, but I have no idea how to setup NFS on ESXi 4. The setup of NFS on FreeNAS is a little different on 8.1 and 8.3, but I would be very surprised if ESXi would give you any hassle with either. If your are starting out from scratch, you should use the latest versions.
A copy of ghettoVCB. Grab a copy from Github (https://github.com/lamw/ghettoVCB) Simply download the hole repository as a single zip file.
Using WinSCP (http://winscp.net) is a great help, but not necessary. Putty is also a great option. Use what you are most comfortable with.
If things get ugly, vmware community is possibly the best place to turn to. (http://communities.vmware.com/docs/DOC-8760) For more details and options customizing the script, please also have a look at the community link and ESXi and FreeNAS needs to be on same subnet.
2. NFS on FreeNAS
What you need to do: Decide on a volume for storing your VM backups on FreeNAS. Start with something small and work your way up. Create a ZFS dataset on this volume. Allow NFS service and finally allow NFS sharing on the ZFS dataset.
Start out by testing if FreeNAS is able to see the ESXi host (optional). Log into FreeNAS. Click 'Shell' from the FreeNAS tabletop - right above 'reboot' in the menu to the left. Try ping [IP address of ESXi host]. I.e. 'ping 172.20.5.175'. You shouldn’t get any packets loss. Otherwise something is wrong, and you need to solve this network issue before anything else.
From the FreeNAS tabletop click 'Storage'. Find a volume you will use for the actual backups and add a ZFS dataset to this volume. Something big is usually good. In the screenshot below I'm using a volume called 'ada2' and adding a ZFS dataset on this volume. Unless you know what you are doing, only set the name of the ZFS dataset. For this test I'm using a dataset called 'GHETTO'. Use a meaningful and short name.
Allow the NFS service on the FreeNAS box: From the FreeNAS tabletop click 'Services'. Find the NFS service on the list and click the little green spanner to the right. Set numbers of servers to 4 and allow asynchronous mode. Make sure the NFS service is set to 'ON'.
Allow NFS to share the ZFS dataset: From the FreeNAS tabletop click 'Sharing'. Click 'UNIX (NFS)' and click 'Add UNIX (NFS) share'. Please ignore the NFS share already created. Comment: Name for this NFS share. Keep it simple. Path: Click the 'Browse' button and navigate to the ZFS dataset you just created. In my case: /mnt/ada2/GHETTO. Or /mnt/[Volume]/[ZFS dataset]. Authorized network or IP addresses: Use the IP address of the ESXi host or leave it empty for test. If used by more hosts, set a netmask instead. FreeNAS/NFS should be okay now.
3. Setup ESXi for the NFS store
You will now add the NFS datastore to the ESXi host directly from the vShere Client. Open the vSphere Client and log into your ESXi host. Click 'Configuration' and then 'Storage' in the hardware box to the left. Now click 'Add storage'.
Choose 'Network File System'. Server: Enter IP address of the FreeNAS box. Fx. 172.20.5.175. Folder: Use the path value for the NFS share. Previous the NFS share was configured to point to: /mnt/ada2/GHETTO. You should properly recheck the NFS share on FreeNAS for the directory.
If ESXi complete the task successfully, the ESXi host will have attached a new datastore. Go check it out: Right click on the (NFS) datastore. Choose 'Browse datastore'. Create some directories. Upload/delete file and make sure everything is OK. Default you will have write access to this datastore.
On the newly added datastore, create a new directory for the actuals backups and make sure you know there it is. Similar when testing the datastore: Right click the datastore and choose 'Browse datastore'. Create a new folder (Use the folder icon with the little green plus sign). Give the folder a simple and meaningful name. Make a note of the datastore name and directory you just created. In my case I have added s datastore called 'GHETTO' in which I have created a directory called 'ghettovms'.
4. Setting up GhettoVCB on the ESXi
We will be running a script directly on the ESXi host. To be able to do that, we need access to the host, and we gain access with SSH. SSH is not running default on a ESXi host - the service must first be allow and then started. You configure the SSH service either from the console or from the vSphere client. If your are running keyboard and stuff directly on the ESXi server, the console is the way to go, but otherwise simply start up the vSphere Client and log into your ESXi host. Once in the vSphere client first select the Configuration tab and secondly the Security Profile.
Now open the Services Properties.
Find SSH service (or daemon) and click Options.
You properly want the service to start up with the host. For testing and the occasional backup, manual will do fine. Remember to start up the service and make sure status is 'Running'. Okay - we can access our ESXi host from Putty or WinSCP now. Try connecting with either WinSCP or putty.
5. Copy and extract the ghetto zip to your ESXi host.
We need to find a (persistent) place where the script, conf and log files eventually will be stored. I will use a local datastore for this. i.e. not the FreeNAS drive. If you are running your ESXi host without any local disks at all, your FreeNAS network drive should work just fine. From the vSphere client go to the Configuration tab. Choose Storage in the left hardware box. The datastores for the ESXi host will be listed. Click on a local datastore on the list (or whatever drive you feel will work out for the best) When selecting the datastore, details for the datastore will be available (just beneath) It's the location of the datastore you need. It will be some nonsense like: /vmfs/volumes/4bfa970f-5dcfb86c-447a-001b215d8dfd. Whatever name you have given your datastore, it will not be visible here. Now - we know where to copy ghettoVCB.
Log into your ESXi host using WinSCP. Host name is your ESXi host ip adresse. You should see something like this. It's very interesting and a lot of things can go horribly wrong if you mess around in here - so don’t. Open the vmfs directory. The vmfs directory just contains two directories: devices and volumes. Open the volumes directory.
You should be able to recognize your ESXi datastore from the directories listed. You will also notice the logical link: datastore1. i.e. my identification for the datastore in the vSphere client. As you have figured out by now: datastore1 and 4bfa970f-5dcfb86c-447a-001b215d8dfd are exactly the same thing. The name of your datastore identification will of course be different.
Now open the correct datastore directory. I my case '4bfa970f-5dcfb86c-447a-001b215d8dfd'. If you are running any VM's from this datastore, you should be able to recognize the structure. A minor detour: Open vSphere Client. Goto Configuration and choose Storage in the hardware box to the left. Right click on the datastore and choose 'Browse datastore'. You should see the exact same file structure as you have been looking at from WinSCP. View from vSphere Client datastore at the top and view from WinSCP at the bottom. Same thing.
You could argue for not mixing up your VM's with scripts, config and log files. i.e. going for a different location or datastore for the ghetto script. You should properly do just that, when you feel comfortable with all this.
You need at least 2 different directories here: One for the script itself and one for log files. From WinSCP create a directory called 'ghettoVCB' and one called 'Log'. The actual backup directory have already been created.
In WinSCP right click on both directories in turn, and choose 'File Names' -> 'Copy URL to clipboard'. The result should be something like this: sftp://bishop/vmfs/volumes/4bfa970f-5dcfb86c-447a-001b215d8dfd/ghettoVCB/ (sftp://[host name]/vmfs/volumes/[datastore]/ghettoVCB/).
Extract ghettoVCB-master.zip locally and copy the files to the ghettoVCB directory with WinSCP. The ghettoVCB directory on your ESXi host should now include: ghettoVCB.conf, ghettoVCB.sh, ghettoVCB-restore.sh, ghettoVCB-restore_vm_restore_configuration_template, ghettoVCB-vm_backup_configuration_template and README. From Putty you should see something like this (dates will differ)
You must set execute permissions for both ghettoVCB.sh and ghettoVCB-restore.sh. In WinSCP Navigate to the script dir on the ESXi host and the properties like below (Right click and choose properties. Make sure they are both 755)
6. Configure GhettoVCB
For the actual configuration bit I will use an example: Backup two running computers to the FreeNAS box. Please double and recheck your own paths with WinSCP.
a) ghetto script is running from (script directory):
/vmfs/volumes/4bfa970f-5dcfb86c-447a-001b215d8dfd/ghettoVCB (or /vmfs/volumes/datastore1/ghettoVCB)
b) ghetto script log is going to (log directory):
/vmfs/volumes/4bfa970f-5dcfb86c-447a-001b215d8dfd/Log/ (or /vmfs/volumes/datastore1/Log/)
Log file is: ghettoVCB.log
c) Computers to backup:
I use a CentOS 6 ([datastore1] 189-CentOS/189-CentOS.vmdk) and a Ubuntu Server ([datastore1] 199-ubuntu/199-ubuntu.vmdk) for this test. This is stupid: Use something small for testing.
Create a new file in notepad called: 'vms_to_backup'. This file must include the above centos and ubuntu vm's, each on a separate line. Like:
189-CentOS
199-ubuntu
Save the file in the script directory (/vmfs/volumes/datastore1/ghettoVCB). To keep things simple, use names like the above VM's without spaces in either filename or directory name.
d) Place to keep the backups (the volume on FreeNAS): /vmfs/volumes/c817db1d-d0c0dd7f/ghettovms (or /vmfs/volumes/GHETTO/ghettovms)
That is vmfs/volumes/[Name of FreeNAS NFS store]/ghettovms
e) NFS server ip
IP adress of the FreeNAS box: 192.168.1.184
f) NFS share
The NFS share should be identically with the path value for the NFS share (/mnt/ada2/GHETTO)
g) NFS ESXi datastore navn
The name for the NFS datastore locally in the vSphere client: GHETTO
h) Backup directory on the NFS datastore
The name of the directory on the NFS datastore: ghettovms
Log in to the ESXi host with WinSCP.
1) Create a new directory on your local computer. Copy the contens for the script directory into this directory. This will be the local script dir (for test and mocking about)
2) Copy the file created in c) vms_to_backup to the script dir on the ESXI host.
3) In local script dir: Make a copy of ghettoVCB.conf ( -> ghettoVCB.conf.org) for safekeeping.
4) And now finally for the script. First make a solid backup, and later start looking at the more fancy stuff. For a more detailed description, please look at the community link above.
VM_BACKUP_VOLUME= (d) /vmfs/volumes/GHETTO/ghettovms
DISK_BACKUP_FORMAT=thin
VM_BACKUP_ROTATION_COUNT=3
POWER_VM_DOWN_BEFORE_BACKUP=0
ENABLE_HARD_POWER_OFF=0
ITER_TO_WAIT_SHUTDOWN=3
POWER_DOWN_TIMEOUT=5
ENABLE_COMPRESSION=0
VM_SNAPSHOT_MEMORY=0
VM_SNAPSHOT_QUIESCE=0
ALLOW_VMS_WITH_SNAPSHOTS_TO_BE_BACKEDUP=0
ENABLE_NON_PERSISTENT_NFS=0
UNMOUNT_NFS=0
NFS_SERVER= (e) 192.168.1.184
NFS_MOUNT= (f) /mnt/ada2/GHETTO
NFS_LOCAL_NAME= (g) GHETTO
NFS_VM_BACKUP_DIR= (h) ghettovms
SNAPSHOT_TIMEOUT=15
EMAIL_LOG=0
EMAIL_SERVER=auroa.primp-industries.com
EMAIL_SERVER_PORT=25
EMAIL_DELAY_INTERVAL=1
EMAIL_TO=
EMAIL_FROM=root@ghettoVCB
WORKDIR_DEBUG=0
VM_SHUTDOWN_ORDER=
VM_STARTUP_ORDER=
Start with these five values and work your way forward from these. Save the customized ghettoVCB.conf file locally and copy it to the EXSi script dir.
Now your ESXi script should include 2 new files: Your version of ghettoVCB.conf and vms_to_backup (The list of VM's to backup).
7. Finally running (or debugging) the script
For this thing, i will run the ghettoVCB script with the f, g and l parameter.
-f List of VMs to backup
-g Location of the conf file
-l Log file.
Make sure you have the correct versions where you want them, and finally navigate to the script directory and run ghettoVCB directly from the script dir on the host.
./ghettoVCB.sh -f vms_to_backup -g /vmfs/volumes/datastore1/ghettoVCB/ghettoVCB.conf -l /vmfs/volumes/datastore1/Log/ghettoVCB.log
You should always start your configuration with the dryrun option. You run the script, but no actual backup will be done. Especially for the “ERROR: failed to locate and extract VM_ID for” the dryrun option is helpful - and you will get a lot of these. the option is: -d dryrun. And you will simply run it like:
./ghettoVCB.sh -f vms_to_backup -g /vmfs/volumes/datastore1/ghettoVCB/ghettoVCB.conf -l /vmfs/volumes/datastore1/Log/ghettoVCB.log -d dryrun
After a little debugging your backups will start appearing on the FreeNAS. While waiting, try open the vSphere Client (Make sure the host is chosen) Choose Performance. On the top right dropdown, choose Network. You should see something like 3500-4000 KBps. That is not super fast at all and should be somewhat faster. The End (Still missing: Running on a schedule and testing restore)