Last week I got busy with upgrading four VDP appliances for a customer. I’ve come across quite some challenges and issues during this task that I’d like to share these with you in case you are facing the same task.
Task overview
So I’m upgrading four appliances, locating in 4 different regions (Remote Offices). These appliances are running VDP 6.0 and need to be upgraded to VDP 6.1.5 (latest and final release to support vSphere 6.5). The reason for upgrading is that the customer wants to upgrade to vCenter Server 6.5, which requires VDP 6.1.3 or higher.
The Future of VDP
Releases after vSphere 6.5 will no longer support VDP, so it might be a good idea to already have a look around at different backup solutions. They’re probably not going to be free like VDP but there might be some cheap options out there (I’ll try to find some time to do some digging into this and publish an article about it).
#LearningModeOn
All right here we go!
Key Takeaway #1: Fresh install or upgrade
My biggest key takeway during this task is that a fresh install is sooooo much faster than upgrading. Especially if you’ve got several upgrades like me (6.0.2.4 to 6.1 and 6.1 to 6.1.5) and a small number of jobs (2 backup jobs and 2 replication jobs per VDP appliance), it’s worth thinking about deploying new VDP appliances.
When deploying a new appliance, you can still link it to existing VDP storage disks. So you don’t lose your previous backup data.
If you are still unsure of what to do, please read on.
Key Takeaway #2: Documentation
It’s very important to document your VDP installation/configuration during the initial set up (even better to do it up front so you can simply perform a next-next finish installation as you have all the details in place).
Think about the appliance VM name, IP address, subnet mask, default gateway, DNS addressess. But also the backup jobs, replication, notification and of course credentials.
You will find out why I was happy this was all documented in the following key takeaways.
Key Takeaway #3: HCL
Before performing any upgrade of a VMware product, make sure you validate the approach using the HCL (Hardware Compatibility List) or VCG (VMware Compatibility Guide).
I usually navigate to it by using http://www.vmware.com/go/hcl but there are other ways to get there.
Using this interface you can see compatbility of software with certain hardware (or vice versa) but also the supported upgrade paths and interoperability between VMware products.
In my case, I noticed that vCenter Server 6.5 and U1 require VDP 6.1.3 or higher. As VDP 6.1.5 is the latest version at moment of writing, I decided this would be the best version to go to. You can see this on the VCG by clicking here.
In the VCG, you can also see what upgrade steps are required when going from a specific version. For me, I had to go from 6.0.2.4 to 6.1 and finally to 6.1.5 (2 upgrades). Click here to see this in an overview.
Key Takeaway #4: Snapshots
Before performing the actual upgrade, make sure you create a snapshot of the VDP appliance. The appliance is already configured to exclude data disks from the snapshot so you only include the OS disk.
If something goes wrong, you can revert to the snapshot and start running the older version. This is what the documentation says. However, in my case the revert to snapshot only made things worse.
There’s a rollback feature inside VDP that rolls back the complete appliance to a certain point in time, rolling back configuration and backup data. This didn’t repair my appliance as well.
My backup and replication jobs were gone and I was forced to redeploy the VDP appliance. That’s why I was very happy with the available documentation as it described how the jobs were created step by step.
Key Takeaway #5: Known issues
There’s a couple of known issues, which are mentioned in the release notes of every VDP release. One in particular, I would like to spend a little time on.
When upgrading an existing VDP appliance, you need to connect the upgrade ISO and use the admin interface to perform the upgrade. After connecting the ISO, the screen should automatically show you an option to upgrade. In my case, the page displayed that there was no ISO connected.
This is a known issue, and there’s a known fix for it as well, which you can get here. I think it’s a bit strange that such fixes are not rolled into the existing VDP deployment packages so you don’t need to manually download these fixes. But maybe that’s easier said then done.
When you install the fix, your ISO should be detected now. If not, you can try to run mount /dev/sr0 /mnt/auto/cdrom using SSH on your VDP appliance. It should definitely be detected now.
Key Takeaway #6: Bandwidth and pre-loading
The customer I was performing these upgrades for, had very limited bandwidth between their primary data center and the remote locations where VDP was running.
Because of this, I downloaded the OVA file of VDP locally to these locations (internet connection was faster) and used OVFtool to deploy VDP to the local ESXi host.
In case you are interested in doing this, make sure you install OVFtool on a machine which is local or close to the ESXi host you want to deploy VDP to (or any other OVF). Next, use the following one-liner to perform the deployment:
ovftool.exe –acceptAllEulas -ds=”datastorename” -dm=thin –name=”vmname” –net:”Isolated Network”=”virtualnetworkname” –prop:”vami.gateway.vSphere_Data_Protection_6.1″=”defaultgateway” –prop:”vami.DNS.vSphere_Data_Protection_6.1″=”dnsip1,dnsip2” –prop:”vami.ip0.vSphere_Data_Protection_6.1″=”vdpipaddress” –prop:”vami.netmask0.vSphere_Data_Protection_6.1″=”netmask” C:TempvSphereDataProtection-6.1.ova vi://vcenterfqdn/datacentername/host/hostname/
Please make sure you edit the bold sections to match your environment and preferences.
The deployment took me a few minutes, compared to running it from a remote location for several hours.
An alternative to this would be deploying using the vSphere Web Client from the local network, but in my case I didn’t have this option.
Key Takeaway #7: Backup jobs, replication and notifications
Both when upgrading and deploying a new appliance, make sure all backup jobs, replication jobs and notifications are in place. Also make sure that timezone settings are correctly configured and notification schedules are OK.