In Azure DevOps, is it possible to add a Test Plan with all Test Suites to a build pipeline (YAML) automatically?

Question

I use Azure DevOps Test Plans and Test Suites to execute automated tests in a YAML build pipeline. For each release I create a new Test Plan with new Test Suites. Actual I search manually for IDs of Test Plan and Test Suites and copy them to the YAML file.

- task: VSTest@2
  displayName: 'Run automated UI tests'
  inputs:
   testSelector: testPlan
   testPlan: 585
   testSuite: 586,929,930,680,683,684,685,931,681,686,687,688,767,682,689,690,691,768,692
   testConfiguration: 2
   uiTests: true
   testRunTitle: 'Automated UI testing'

Is there a possibility to do that automatically? Or a possibility to reduce the manual effort e.g. just change the Test Plan ID in Pipeline and all Test Suites are automatically included?

Answer 1

The YAML file is in your repository. So you can edit this file through REST API. Here is an example: Update a file. In this case, you may have some template of your YAML file, identify Test Plan ID and Test Suites (Get a test suites for plan), then update the YAML file with new IDs.

Answer 2

Is there a possibility to do that automatically? Or a possibility to reduce the manual effort e.g. just change the Test Plan ID in Pipeline and all Test Suites are automatically included?

You can get the test suites for the test plan through the script in a Powershell task, and then assign the obtained result to a variable.

Use Test Suites - Get Test Suites For Plan rest api:

GET https://dev.azure.com/{organization}/{project}/_apis/testplan/Plans/{planId}/suites?api-version=6.0-preview.1

Sample script:

$url = 'https://dev.azure.com/{organization}/{project}/_apis/testplan/Plans/{planId}/suites?api-version=6.0-preview.1';

$response = Invoke-RestMethod -Uri $url -Headers @{Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"} -Method Get

$testSuites = $response

Write-Host "results = $($testSuites | ConvertTo-Json -Depth 100)"

In VSTest task:

- task: VSTest@2
  displayName: 'Run automated UI tests'
  inputs:
   testSelector: testPlan
   testPlan: 585
   testSuite: $(testSuites)
   testConfiguration: 2
   uiTests: true
   testRunTitle: 'Automated UI testing'

Answer 3

Here's a full solution that works as of 3/1/2022 using Azure DevOps API

- pwsh: |
    $organization = "YOUR ORG NAME HERE"
    $project = "YOUR PROJECT NAME HERE"
    $planId = 585
    $password = ConvertTo-SecureString -String $env:SYSTEM_ACCESSTOKEN -AsPlainText -Force

    $url = "https://dev.azure.com/$organization/$project/_apis/test/Plans/$planId/suites?api-version=5.0"
    $cred = New-Object –TypeName "System.Management.Automation.PSCredential" –ArgumentList "AzPipeline", $password

    $response = Invoke-RestMethod -Uri $url -Authentication Basic -Method Get -Credential $cred

    $testSuites = $response |
      ForEach-Object{ $_.value.id} |
      Join-String -Separator ', '

    Write-Host "##vso[task.setvariable variable=testSuites;]$testSuites"
  env:
    SYSTEM_ACCESSTOKEN: $(System.AccessToken)

- task: VSTest@2
  inputs:
    testSelector: testPlan
    testPlan: 585
    testSuite: $(testSuites)
    testConfiguration: 2
    uiTests: true
    testRunTitle: 'Automated UI testing'