Limitation of Concurrent Tasks
When you start a data protection or disaster recovery job, Veeam Backup & Replication analyzes the list of VMs added to the job, and creates a separate task for every disk of every VM to be processed. Veeam Backup & Replication then defines what backup infrastructure components must be used for the job, checks what backup infrastructure components are currently available, and assigns necessary components to process the created job tasks.
Backup infrastructure components typically process several tasks at the same time. You can limit the number of tasks that backup infrastructure components must process concurrently. Task limitations helps you balance the workload across the backup infrastructure and avoid performance bottlenecks.
Veeam Backup & Replication lets you limit the number of concurrent tasks for the following backup infrastructure components:
Task limits set for backup infrastructure components influence the job performance. For example, you add a VM with 4 disks to a job and assign a backup proxy that can process maximum 2 tasks concurrently for the job. In this case, Veeam Backup & Replication will create 4 tasks (1 task per each VM disk) and start processing 2 tasks in parallel. The other 2 tasks will be pending.
How Task Limitation Works
Task limiting is performed by the Veeam Backup Service. The Veeam Backup Service is aware of all backup proxies and backup repositories in the backup infrastructure, and task limitation settings configured for them.
When a job starts, it informs the Veeam Backup Service about the list of tasks created for the job, and backup infrastructure resources that must be used for the job. The Veeam Backup Service detects the number of tasks that required backup infrastructure components are currently processing, and analyzes the number of allowed tasks for these components. If the number of currently processed tasks has reached the allowed limit, the backup infrastructure component will not start processing a new task until one of the currently running tasks finishes.
To limit the number of concurrent tasks on a backup proxy, you must define the Max concurrent tasks setting for the backup proxy. For onhost backup, you must define the Task limit setting for the source Microsoft Hyper-V host.
The maximum number of concurrent tasks depends on the number of CPU cores available on the backup proxy. It is strongly recommended that you define task limitation settings using the following rule: 1 task = 1 CPU core. For example, if a backup proxy has 4 CPU cores, it is recommended that you limit the number of concurrent tasks for this backup proxy to 4.
To limit the number of concurrent tasks in a backup repository, you must enable the Limit maximum concurrent tasks to <N> option on the backup repository and define the necessary task limit.
The maximum number of concurrent tasks depends on the number of CPU cores available in the backup repository. It is strongly recommended that you define task limitation settings using the following rule: 2 tasks = 1 CPU core. It is recommended to configure 2 GB RAM for each concurrently processed machine or file share. In case of shared folder backup repositories, the same amount of resources is required for gateway servers. For example, for 1 VM with 4 virtual disks you need 2 CPU cores and 2 GB RAM. For 2 VMs with 2 virtual disks you need 2 CPU cores (as the number of tasks was not increased) and 4 GB RAM.
Synthetic operations performed in the backup repository (such as synthetic full backup, backup files merge and transformation) are also regarded as tasks. The number of tasks performed during these operations depends on the type of backup chains stored on the backup repository:
- For regular backup chains, Veeam Backup & Replication creates 1 task per job.
- For per-machine backup chains, Veeam Backup & Replication creates 1 task per every VM chain (that is, every VM added to the job).
If you use backup repositories for backup copy jobs, you must also consider tasks for read operations.
When you limit the number of tasks for the backup repository, bear in mind the storage throughput. If the storage system is not able to keep up with the number of tasks that you have assigned, it will be the limiting factor. It is recommended that you test components and resources of the backup infrastructure to define the workload that they can handle.
Task Limitation for Components with Several Roles
One machine can perform several roles. For example, you can assign roles of the backup proxy and backup repository to the same machine, or use a backup proxy as a gateway server for a shared folder backup repository. In such situation, you must make sure that the backup infrastructure component is able to process the cumulative number of tasks specified for different roles.