How CDP Works
This section describes how CDP works during replication. To learn how CDP works during data recovery, see Failover and Failback for CDP.
CDP workflow during replication is divided into two parts: backup infrastructure component configuration and data transfer.
During the configuration, Veeam Backup & Replication configures the required backup infrastructure components. Veeam Backup & Replication also reconfigures the components if something changes in the infrastructure. During data transfer, Veeam Backup & Replication creates short-term and long-term restore points by sending disk data blocks and changes made to them. For more information on restore points, see Replication Chain.
Backup infrastructure component configuration and data transfer are constant processes.
The following steps apply during the initial configuration, that is, when a CDP policy starts for the first time or after the policy is enabled:
- Veeam CDP Coordinator Service reads policy settings from the configuration database and creates a list of VM tasks to process. For every VM added to the CDP policy, Veeam Backup & Replication creates a new task.
- Veeam CDP Coordinator Service checks that required backup infrastructure components are available.
- Veeam CDP Coordinator Service queries information about VMs added to the CDP policy and virtualization hosts from the vCenter Server.
- Veeam CDP Coordinator Service requests vCenter Server to create on the target host empty VM replicas with the same configuration as original VMs, but with empty virtual disks. vCenter Server registers the created VM replicas.
- Veeam CDP Coordinator Service requests vCenter to apply the Veeam CDP Replication storage policy to virtual disks of VMs on the source ESXi hosts. This storage policy adds the component that is required for CDP and that gets data of all I/O operations. For more information on storage policies, see VMware Docs.
The Veeam CDP Replication storage policy itself is created on the vCenter Server when you install the I/O filter. For more information on how to install the filter, see Installing I/O Filter.
- Veeam CDP Coordinator Service selects which VMware CDP proxies will be used for data transfer and sets a number of rules for data transfer, such as network traffic throttling rules and so on.
If you select automatic proxy selection when configuring the CDP policy, Veeam Backup & Replication analyzes the current workload on CDP proxies and selects a VMware CDP proxy according to the following priority rules (starting from the most preferable one):
- VMware CDP proxy on a physical machine
- VMware CDP proxy on a VM located on the same ESXi host — that is, the source proxy on the host where original VMs are located, the target proxy on the host where VM replicas are located
- Other VMware CDP proxies
For more information on how to specify the proxy selection mode, see Specify Data Transfer and Replica Settings.
- Veeam CDP Coordinator Service sends to the backup infrastructure components configurations required for CDP. This configuration includes such information as RPO, short-term and long-term retention settings.
After the initial configuration finishes, Veeam Backup & Replication starts monitoring the backup infrastructure. If something changes in the infrastructure or CDP policy settings, Veeam Backup & Replication reconfigures the components. Consider the following examples:
- If a VMware CDP proxy becomes unavailable, Veeam CDP Coordinator Service on the backup service gets a notification that this proxy is no longer available. Then, the service selects another proxy.
If you add new proxies to the backup infrastructure, Veeam Backup & Replication does not use these proxies for the already created CDP policies — that is, Veeam Backup & Replication does not reconfigure the infrastructure. If you have selected automatic proxy selection for a CDP policy and want to use the newly added proxies, disable and then enable the CDP policy. If you have selected the proxies manually, edit the CDP policy settings and add the required proxies.
- If virtual disks are added to original VMs, Veeam CDP Coordinator Service requests vCenter to create the disks on the target host, applies the storage policy and selects VMware CDP proxies to transfer disk data.
- If new VMs are added to the CDP policy, Veeam CDP Coordinator Service requests vCenter to create VM replicas with empty disks, applies the storage policy to the VM disks and selects which VMware CDP proxies to use for data transfer.
- If the original VMs were migrated to other host or datastore using VMware vSphere vMotion, Veeam CDP Coordinator Service analyzies how data will be transferred after the migration and selects VMware CDP proxies to optimize data transfer.
Component reconfiguration requires Veeam CDP Coordinator Service to be working. If the coordinator goes down, existing CDP policies still work, create and remove short-term restore points. Long-term restore points are not created and removed because it is the coordinator who manages them. However, if any of the components goes out of service, for example, VMware CDP proxy goes offline or VMware vSphere vMotion changes the infrastructure, CDP policies start failing until Veeam CDP Coordinator Service is repaired.
Data transfer starts right after Veeam Backup & Replication configures the backup infrastructure components (performs the initial configuration). Data transfer differs during the initial synchronization and during the incremental synchronization.
As a rule, the initial synchronization is performed when disk data is sent to the target host for the first time. During the initial synchronization, Veeam Backup & Replication sends data for full copies of virtual disks and creates the very first restore points.
During the incremental synchronization, Veeam Backup & Replication mainly sends data for incremental changes made to virtual disks and creates short-term and long-term restore points. For more information on restore points and files created for VM replicas, see Replication Chain.
Data Transfer Algorithm During Initial Synchronization
The following steps apply to data transfer during the initial synchronization:
- On the source host, I/O filter reads all data from VM disks and sends it to the source VMware CDP proxies.
As the original VMs are still running, data for the already transferred data blocks can change. I/O filter intercepts these changes and sends them to the proxies. Sending changes instead of whole changed data blocks helps minimize traffic sent over the network.
- The Veeam CDP Proxy Service on the source proxies compresses, encrypts the received data and sends it to the target proxy.
- The Veeam CDP Proxy Service on the target proxies decompresses and decrypts the received data. Then sends data to the target host.
- The I/O filter on the target host saves the received data to virtual disks. The saved data relates to the very first long-term restore point. This restore point is crash-consistent.
Veeam Backup & Replication starts creating long-term and short-term restore points only after the initial synchronization finishes.
After the initial synchronization finishes, Veeam Backup & Replication starts the incremental synchronization.
Data Transfer Algorithm During Incremental Synchronization
During the incremental synchronization, Veeam Backup & Replication creates short-term and long-term restore points. To create short-term restore points, Veeam Backup & Replication intercepts changes made by transactions on VM disks and sends these changes to the target datastore. Changes are constantly transferred and are saved to transaction logs on the target datastore. To create long-term restore points, Veeam Backup & Replication uses data of short-term restore points and saves the restore points to delta disks. Long-term restore points are created by schedule.
The following steps apply when Veeam Backup & Replication transfers data for short-term restore points:
- On the source host, I/O filter intercepts data of all I/O operations and sends this data to the source VMware CDP proxies.
- Once in the RPO, the Veeam CDP Proxy Service on the source proxies prepares data required for a short-term restore point. For this, Veeam CDP Proxy Service gets the latest state of the data that the source VMware CDP proxies has accumulated.
- The source Veeam CDP Proxy Service compresses, encrypts data and sends it to the target proxy.
- The target Veeam CDP Proxy Service decompresses and decrypts the received data. Then sends data to the target host.
- The I/O filter on the target host saves the received data to transaction logs.
The following steps apply when long-term restore point creation is scheduled:
- [If application-aware image processing is enabled for the CDP policy] Veeam Backup & Replication connects to VM guest OSes, deploys non-persistent runtime components or connects/deploys persistent agent components on VM guest OSes and performs in-guest processing tasks such as quiescing applications on the VM and creating a consistent view of application data. For more information on guest agent and non-persistent components, see Non-Persistent Runtime Components and Persistent Agent Components.
- On the target datastore, the I/O filter forms a long-term restore point using data of short-term restore points created since the creation of the previous long-term restore point. Data for the new long-term restore point is saved to a delta disk.