How to organize the execution of each test on a separate virtual machine?

Question

I would like to ask the public how to organize complex integration tests when you need to start a virtual machine to run each test. First, let me describe how organized tests we have now and what I do not like this. The process is organized using TFS Process Templates and Custom Actions.

1. First step: On the build agent is assembled the project.
2. From the custom action (using vmware api) run the virtual machine.
3. Copy to the virtual machine the assembled project dlls.
4. Run mstest.exe with the necessary parameters on the virtual machine
5. Copy Test Results to the build agent and analyze it manually in the custom action.

So, steps 2-5 we repeat for each test.

Disadvantages of this approach - complexity. We need to manually analyze the test results, if the format of these files will changes - we get a lot of code rewriting. It would be much better if the TFS build agent was a virtual machine and TFS will revert it themself before running each test. How do you organize such kind of tests. Specifically, in this case we are talking about CodeUI, but similar problems occur frequently. Maybe my question is too general, but any ideas will be very helpful...

Answer 1

If you were to move to Lab Management and Release Management much of what you are trying to do is out of the box with Visual Studio ALM. You can create and build out environments either in Hyper-V or in Azure dynamically.

I would recommend using Release Management and Powershell to orchestrate this. The RM team have a good demo:

http://blogs.msdn.com/b/visualstudioalm/archive/2014/11/11/deploying-and-testing-web-applications-using-release-management.aspx

Here they both deploy the application and then run codedUI.