Running automated UI tests as part of CI

Question

I'm working on an automated test suite for a large WPF application. I've created a wrapper library for Microsoft Windows Automation API (previously known as UI Automation Framework), and I use it in order to implement various sanity tests. My team would like to do Continuous Integration with these tests by running them every check-in. We're working with TFS (2012 I believe) and MSTest.

The problem with these tests is that they require the UI to be up and running to actually run the tests. I guess the idea is to copy the built binaries to a remote VM and run the tests with the Remote Agent, but I'm not entirely sure on where to begin.

Has anyone ever done this before and has any code / build workflow XAML examples for achieving something like this?

Answer 1

You need to set up a standard environment on the test machines, which will install a test agent on them. The test agent then runs the tests for you, on your UI.

This should get you started: http://msdn.microsoft.com/en-us/library/ee390842.aspx

Answer 2

For my CI server, I use TeamCity by JetBrains. With it, you can setup multiple build agents to connect to the TeamCity server quite easily.

Specifically with regard to testing using UI Automation, you will need to have an interactive desktop session on all of your build agents as you mentioned. To do so, I set them all to auto-login and I have a startup script that runs C:\BuildAgent\bin\agent.bat start so the TeamCity agent starts in the interactive session.

TeamCity hooks in quite nicely to TFS and should be able to run your tests easily. I use NUnit as my test runner, but I can't imagine it being terribly difficult to get running. Best of all, it is free for three build agents and up to 20 build configurations.

Hope this helps.