How to make and control of parallel execution of Azure DevOps Pipeline?

Question

I am using Windows Self hosted agent for my Azure DevOps pipelines. Currently the pipelines are executed sequentially. If more than one pipelines triggered from different ADO projects, then it has to wait in queue to get the agent. In order to execute the pipeline in parallel, I came to know from some tutorials if we increase the paid parallel jobs for self hosted agent under billing section of Organization setting. Is my understanding correct? If so what are the precautionary steps I need to take. Do we have any control of when the pipelines to be executed in parallel?

Thanks.

Answer 1

In order to run self-hosted parallel jobs, you need to purchase parallel jobs and register several self-hosted agents.

For parallel jobs, you can register any number of self-hosted agents in your organization. If you want to run 3 jobs in parallel, then you must register at least 3 self-hosted agents in one agent pool. DevOps charges based on the number of jobs you want to run at a time, not the number of agents registered. There are no time limits on self-hosted jobs. For private projects, you can have one job and one additional job for each active Visual Studio Enterprise subscriber who is a member of your organization.

About how to purchase parallel jobs, please refer to Buy parallel jobs.

For how to control the use of parallel jobs, please refer to the following:

1. For classic pipeline, you can specify when to run the job through dependencies and Run this job in Additional options in the agent job. Then the pipeline will run in sequence according to your settings.

2. For YAML pipeline, you can specify the conditions under which the job should run with "dependsOn" and "condition". For example:

For more info about conditions, please refer to Specify conditions

If you don't specify a specific order, the jobs will run in parallel based on the parallel jobs you purchased.

Answer 2

I don't know if my experience can help. I'll try. I started a new job and we use self-hosted TFS / Azure DevOps. I am changing our build process to create 3 product SKUs (it uses conditional compilation). Let's call them Good, Better & Best.

I edited the Build definition. First I switched to the Variables tab. I created a Process variable named SKUs and set it to Good,Better,Best. The commas are important.

Next I switched to the Tasks tab. I located the Agent Phase. Mine was called Phase 1. Select it. On the right, under Parallelism, I selected Multi-configuration. In the Multipliers text field I entered SKUs. I set Maximum number of agents to 3.

What I don't yet know is the TFS back-end administration and options that the company purchased beforehand.