Background
We are using Azure DevOps Server 2020 at our company. (Version Azure DevOps Server 2020 Update 1.1, installed from here)
We are using the XML process workflow with customized workitems, as well as a yaml pipeline configuration that runs about 100k tests.
Sometimes tests prove flaky. We previously used teamcity where you can simply mark a test flaky and people would recognize that. In DevOps, it is not possible yet. I have checked the settings page and the Test management section is not there.
Instead, we have resorted to the use of Bugs to assign a flaky test to a team that will fix it:
Screenshot: Test failure with the option to create a bug for it
It does however clutter/skew our bug statistics. Ideally, we would like to use Bug workitem for problems reported by humans, not test failures.
What I have tried
What I have tried is to create a different work item type Test Issue in the Microsoft.BugCategory with the intent for it to be used when a test has failed:
<?xml version="1.0" encoding="utf-8"?>
<cat:CATEGORIES xmlns:cat="http://schemas.microsoft.com/VisualStudio/2008/workitemtracking/categories">
<CATEGORY refname="Microsoft.BugCategory" name="Bug Category">
<DEFAULTWORKITEMTYPE name="Bug" />
<WORKITEMTYPE name="Test Issue" />
</CATEGORY>
<CATEGORY refname="Microsoft.RequirementCategory" name="Requirement Category">
<DEFAULTWORKITEMTYPE name="User Story" />
<WORKITEMTYPE name="Robot Story" />
<WORKITEMTYPE name="API Request" />
<WORKITEMTYPE name="Proposal" />
</CATEGORY>
[. . .]
The line of interest is this one: <WORKITEMTYPE name="Test Issue" />
However, this did not make Test Issue appear under the test result's "Create Bug" button.
Questions
Is there a way to configure the button in the screenshot to present more options than simply Create bug?
Is it possible to hack the installation to use a different work item than Bug to be created when using the Create bug button?