Azure DevOps pipelines: job starting and waiting for another job to finish

Question

Is there any analogue to waiting for the triggered build to finish for jobs? Meaning one job starts and is waiting for another job running in parallel to finish.

What I am trying to accomplish is the following. I have self-hosted agents on VMs. The agents on the host are starting, stopping, restoring and so on the VMs. One host agent corresponds to one VM. In some cases the same VM is used in different pipelines and I need to avoid concurrency in this case. Initially I had classical pipelines and used build triggering task and the VM agents were running triggered build while the host agent was waiting for it to finish. This was ensuring that another pipeline can't grab the same VM.

Now there is a job on the host restoring and starting the VM, then dependent job on VM agent and after it's finished another dependent job on the host agent stops VM. While the job on VM is running the host agent is idle and might take another job from another pipeline and mess things up. What is needed is to keep it occupied during the whole pipeline run. What is the best way to accomplish this?

Thanks.

Answer 1

In some cases the same VM is used in different pipelines and I need to avoid concurrency in this case.

According to the above sentence, you want to forbid more than one running job to target a particular VM (environment).

In order to achieve such behavior you can use the deployment job in your pipelines which will target the environment (your VM).

- deployment: deployJob
  environment: vm_a #Particular VM

And then add an Exclusive lock on your environment environment/vm

Below is the working example:

Preparation:

1. Create two pipelines that target the same VM (vm_a):

• pipeline-x

# pipeline-x.yml
trigger: none

pool:
  vmImage: ubuntu-latest

stages:
- stage: deployStage
  lockBehavior: sequential
  jobs:
    - deployment: deployJob
      environment: vm_a
      strategy:
        runOnce:
          deploy:
            steps:
            - script: |
                echo I am occupying vm_a
                sleep 60
              displayName: 'Use VM'

• pipeline-y

# pipeline-y.yml
trigger: none

pool:
  vmImage: ubuntu-latest

stages:
- stage: deployStage
  lockBehavior: sequential
  jobs:
    - deployment: deployJob
      environment: vm_a
      strategy:
        runOnce:
          deploy:
            steps:
            - script: |
                echo I am occupying vm_a
                sleep 60
              displayName: 'Use VM'

2. Create vm_a environment and set exclusive lock on it.

Verification:

1. Run pipeline-x.
2. Wait 30 seconds.
3. Run pipeline-y.

Observe: Until the pipeline-x is running the pipeline-y will wait for the job that targets vm_a to finish.

NOTE: I don't know the details of your jobs, and don't know whether you already use the ADO environment set to Virtual Machine resource type. The environment (vm_a) from the example is the environment with the Resource: none (You can add resources later) option.