Pytest + coverage regression test done when a pull request is submitted (azure pipelines and bitbucket)

Question

I have a python package in a bitbucket repository. I would like to setup azure pipelines so that when I submit a pull request to the master branch of that repository, a bunch of pytest tests are run.

Along with this (either separately, or preferably in the pytest test suite itself) I would like for a coverage test to be run, and that test to fail if the coverage percentage of the master branch is higher than the coverage percentage of the branch that is being merged into it is.

The idea is that a pull request couldn't be merged unless the test coverage increased.

Does anyone know how I might do this?

Answer 1

Pytest + coverage regression test done when a pull request is submitted (azure pipelines and bitbucket)

I am afraid we could not manage the code in the Bitbucket repository, but build in the Azure pipeline at this moment.

We could set the pull request validation to execute pytest tests when we submit a pull request to the master branch of Bitbucket repository:

However, we also need a status check if the coverage percentage of the master branch is higher than the coverage percentage of the branch that is being merged for the pull request. But it seems Branch permissions or Bitbucket does not have this feature.

If we migrate the repo from Bitbucket to Azure DevOps, then we could add a Build Validation to execute pytest tests and add a coverage status check. to check the coverage percentage:

Code coverage for pull requests

Code coverage metrics and branch policy for pull requests

General steps:

• Add a Build Validation to execute pytest tests
• Add a task to get the coverage percentage of the master branch and overwrite the value of coverage target in the azurepipelines-coverage.yml in Azure repo.
• Add a coverage policy.