This is an archive version of the document. To get the most up-to-date information, see the current version.

Data Compression and Deduplication

Veeam Backup & Replication provides mechanisms of data compression and deduplication. Data compression and deduplication let you decrease traffic going over the network and disk space required for storing backup files and VM replicas.

Data Compression

Data compression decreases the size of created backups but affects duration of the backup procedure. Veeam Backup & Replication allows you to select one of the following compression levels:

  • None compression level is recommended if you plan to store backup files and VM replica files on storage devices that support hardware compression and deduplication.
  • Dedupe-friendly is an optimized compression level for very low CPU usage. You can select this compression level if you want to decrease the load on the backup proxy.
  • Optimal is the recommended compression level. It provides the best ratio between size of the backup file and time of the backup procedure.
  • High compression level provides additional 10% compression ratio over the Optimal level at the cost of about 10x higher CPU usage.
  • Extreme compression provides the smallest size of the backup file but reduces the backup performance. We recommend that you run backup proxies on computers with modern multi-core CPUs (6 cores recommended) if you intend to use the extreme compression level.

Data Compression and Deduplication Note:

If encryption is enabled for a job and the Decompress backup data blocks before storing check box is selected in the settings of the target backup repository, Veeam Backup & Replication does not compress VM data. Therefore, in the job statistics, you may observe a higher amount of transferred data (the Transferred counter) as compared to a job for which encryption is disabled. For details on job statistics, see Viewing Real-Time Statistics.

Changing Data Compression Settings

You can change data compression settings for existing backup jobs. After you change the settings, you will not need to create new full backups to use new settings. Veeam Backup & Replication will automatically apply the new compression level to the newly created backup files after you save the settings. Previously created backup files will not be affected.

However, if you use the reverse incremental backup method, the newly created backup files will contain a mixture of data blocks compressed at different levels. For example, you have a backup job that uses the reverse incremental backup method and the Optimal level of compression. After several job sessions, you change the compression level to High. In the reverse incremental backup chains, the full backup file is rebuilt with every job session to include new data blocks. As a result, the full backup file will contain a mixture of data blocks: data blocks compressed at the Optimal level and data blocks compressed at the High level. The same behavior applies to synthetic full backups: synthetic full backups created after the compression level change will contain a mixture of data blocks compressed at different levels.

If you want the newly created backup file to contain data blocks compressed at one level, you can create an active full backup. Veeam Backup & Replication will retrieve data for the whole VM image from the production infrastructure and compress it at the new compression level. All subsequent backup files in the backup chain will also use the new compression level.

Deduplication

Data deduplication decreases the size of backup files. With data deduplication enabled, Veeam Backup & Replication does not store to the resulting backup file identical data blocks and space that has been pre-allocated but not used.

We recommend you enable data deduplication if your backup or replication jobs contain several VMs that have a lot of free space on their logical disks or VMs that have similar data blocks — for example, VMs that were created from the same template. However, note that data deduplication may decrease job performance.

Veeam Backup & Replication uses Veeam Data Movers to deduplicate VM data:

You can change data deduplication settings for existing jobs. After you change the settings, you will not need to create new full backups to enable/disable the deduplication. Veeam Backup & Replication will automatically apply the change to the newly created backup files after you save the settings. Previously created backup files will not be affected.

Storage Optimization

Veeam Backup & Replication uses data blocks of different size to process VMs. Data block size depends on the type of storage you select as a backup target.

When you deduplicate a large backup file to small data blocks, Veeam Backup & Replication produces a very large deduplication metadata table, which can potentially overgrow memory and CPU resources of your backup repository. To optimize the job performance and data block size, you can choose one of the following options in accordance with the size of your backup files.

Storage optimization option

Block size

Description

Local target (large blocks)

4096 KB

Recommended for backup files that are larger than 16 TB.

This option will provide the lowest deduplication ratio and the largest size of incremental backup files.

Local target

1024 KB

Recommended for backup to SAN, DAS or local storage.

This option provides the fastest backup job performance but reduces the deduplication ratio, because with larger data blocks it is less likely to find identical blocks.

LAN target

512 KB

Recommended for backup to NAS and onsite backup.

This option provides a better deduplication ratio and reduces the size of a backup file because of reduced data block sizes.

WAN target

256 KB

Recommended if you are planning to use WAN for offsite backup.

This option provides the maximum deduplication ratio and the smallest size of backup files that allows you to reduce the amount of traffic over WAN.

Changing Storage Optimization Settings

You can change storage optimization settings for existing jobs. New settings will not have any effect on previously created backup files in the backup chain. They will be applied to new backup files created after the settings were changed.

For Veeam Backup & Replication to apply the new settings, use the following instructions.

Backup Jobs

To apply new storage optimization settings in backup jobs, you must create an active full backup after you change storage optimization settings. Veeam Backup & Replication will use the new block size for the active full backup and subsequent backup files in the backup chain.

Backup Copy Jobs

To change data block size for backup copy jobs, you must perform the following actions:

  1. Change data block size in settings of the initial backup job.
  2. Create an active full backup with the initial backup job.
  3. Create an active full backup with the backup copy job.