How to run MTM tests on multiple product builds?

Question

We have MTM tests running on Release build of our product (Desktop Application). Now we want the same tests to run on two product builds: Beta and Release.

When a test run is initiated from MTM (or tcm), we need a way to pass a 'value' to the test run telling it which version/build of the product it needs to test. This 'value' will then be read in the test method and correct decision will be taken while the tests are executing (like installation path, test results file updates etc).

Is there any way to achieve this? in TFS or MTM?

Answer 1

We also faced similar problem in our project. We decided to modify the build definition template to take product build type (Beta or RTM or Release) as an input parameter. Using this value during TFS build, we can either update the TFS build name to reflect the product build type or create a file (xml) as part of TFS build process to contain this type detail.

See here for more detail on how to add Arguments and Parameters to build definition: http://www.ewaldhofman.nl/post/2010/04/27/Customize-Team-Build-2010-e28093-Part-2-Add-arguments-and-variables.aspx

Answer 2

Pls take a look at the below link, if it can be used to suit your needs.

http://blogs.infosupport.com/switching-browser-in-codedui-or-selenium-tests-based-on-mtm-configuration/

one question: Are you using Build-Deploy-Test flow to install the product on the environment or doing it any other way?

Answer 3

So, when you select to run a set of automated tests and pick the build from the drop down list this tells MTM which drop folder to go look in. So if your configuration is code, as it should be, then you can set this up to be automatic.

It is not possible to pass additional variables when you start a test run in MTM.

You could setup your tests to run from the Release Management tool instead. You would then be able to configure the environment however you like based on passed in veriables.

http://nakedalm.com/execute-tests-release-management-visual-studio-2013/

Answer 4

Consider using Test Settings.

If you start an automated tests from MTM you can specify Test Settings to use when running this tests.

In "Advanced" part of Test Settings you can specify scripts to run on your environment before running the tests.

1. Create two scripts, one for Release and one for Beta version. These scripts could create a file with particular content, set an environment variable or do something else that can then be checked by your test, when it’s running.
2. Create two Test Settings, one for Release and one for Beta version and set up appropriate script to run for each Test Settings.
3. Use one of these Test Settings when starting tests.

This way you could pass information to your test.