I’ve recently written several articles on new backup and storage features in vSphere and spent a lot of time trying to learn more about the vStorage APIs and the new Changed Block Tracking (CBT) feature to understand how it works, so I thought I would share that information.
CBT is a new feature in vSphere that can keep track of the blocks of a virtual disk that have changed since a certain point in time. This is extremely useful for backup and replication applications that can use this information to greatly improve incremental backup and replication times. Without CBT these applications have to figure out changed blocks on their own so being able to get this information for free using the vStorage advanced programming interfaces is extremely valuable to them.
CBT is not really part of the vStorage APIs but is a new feature of the VMkernel that is built into the storage stack. The CBT feature can be accessed by third-party applications as part of the vStorage APIs for Data Protection. Applications can use the API to query the VMkernel to return the blocks of data that have changed on a virtual disk since the last backup operation. You can use CBT on any type of virtual disk, thick or thin and on any datastore type except for physical mode Raw Device Mappings. This includes both NFS and iSCSI datastores.
Besides requiring vSphere, a prerequisite for using CBT is that a virtual machine must be using version 7 virtual hardware. While this is the default in vSphere (you can still choose the old version 4 hardware that was used in VMware Infrastructure 3, if you upgraded a host from ESX 3 to ESX 4 you must also upgrade the virtual hardware of the VMs to version 7 to use this feature.
The CBT feature is disabled by default; the reason for this is there is a very small bit of overhead that occurs when using it. However this overhead is a small price to pay for the great efficiencies that happen by enabling it. This feature is not global and can be enabled on only select VMs that you want to take advantage of this functionality. It can be enabled either through the vSphere client or by using the SDK. To enable it in the vSphere client you need to add a configuration parameter to each VM using the below steps:
1. Power off the VM. This is necessary to add a configuration parameter, edit the settings of the VM. Select the Options tab, then under Advanced, General click the Configuration Parameters button.
2. Next, click the Add Row button. You first need to add a general parameter for the VM to enable the feature and then add additional ones for each virtual disk that you wish to enable it on. For the general parameter enter “ctkEnabled” for the name and “true” for the value.
3. For each virtual disk you need to add “scsi#:#.ctkEnabled” for the name where the # signs should be replaced by the controller/disk number for each disk. Usually this is 0:0 for the first disk, then 0:1 for the second disk, etc. Also use “true” for the value for this parameter.
4. Click OK when you are done to save it.
An alternate method to enable CBT is using the SDK. Many backup applications that support CBT will automatically do this for you. For details on using the SDK method see this VMware tech note which describes how to use VirtualMachineConfigSpec and ReconfigVM_Task methods to accomplish this programmatically. Typically you will not want to enable this unless you have a specific application that can utilize this feature such as VMware Data Recovery or Veeam Backup & Replication.
Once enabled a VM must go through what is called a stun/unstun cycle for it to take effect. This cycle happens during certain VM operations including power on/off, suspend/resume, create/delete snapshot. During this cycle a VM’s disk are reopened which allows a change tracking filter to be inserted into the storage stack for that VM. You might wonder where CBT stores the information about changed blocks for a virtual disk, it does this in a special “-ctk.vmdk” file that is created in each VM’s home directory for each virtual disk that it is enabled on.
This size of this file is fixed and does not grow beyond its initial size unless you increase the size of a virtual disk. The size of this file will vary based on the size of a virtual disk which is approximately .5MB for every 10 GB of virtual disk size. Inside this file the state of each block is stored for tracking purposes using sequence numbers that can tell applications if a block has changed or not. One of these files will exist for each virtual disk that CBT is enabled on.
CBT is a great feature that really improves the efficiency and speed of virtual machines’ backup, restore and replication operations in vSphere. Several backup applications have already taken advantage of this new feature and are reporting greatly improved incremental backup times and being able to achieve near continuous data protection because of it.