"Art of Test" for C# Automated Web Testing in CruiseControl.NET

Question

We currently use SWEA (http://webiussoft.com) to run automated tests of the site during our CruiseControl.NET overnight builds.

We are looking for a more robust solution and we are looking at the "Art of Test" (http://www.artoftest.com) solution.

Has anyone experience with this tool?

Most importantly, how easy is it to maintain tests using this tool?

Or are there any other C#/NUnit/CruiseContol.NET solutions that you would recommend?

Answer 1

Another option could be WatiN.

Answer 2

I work for ArtOfTest. Over the past couple of days I've been researching what it takes to get our framework to play nicely with CruiseControl.NET. The main key is to run the CruiseControl.NET server in console mode (instead of as a Windows service). The reason for this is to allow the unit test run phase to be able to interact directly with the desktop, which is required for UI testing. Generally code that runs under a Windows service is not allowed to interact with the desktop (which ours requires) and thus will fail when it tries.

TeamCity is also another good CI server. I'm looking at that right now and have gotten it mostly figured out.

If you have any other more specific questions feel free to contact us at [email protected]. I'll also monitor this thread as well.

WatiN is not bad. Unfortunately it lacks any sort of VisualStudio integration and is missing many other features that we offer.

Thx, Cody

Answer 3

Seems like the two big ones are Watin & Selenium. I haven't done any work with Selenium, so no opinion there.

I've used Watin a fair amount, and it's pretty good, but there are many issues. It can be hard to deal with wildly varying response times from a browser... setting timeout durations and the like.

The latest version of Watin supports IE & Firefox, which is pretty cool.

As far as test maintenance, in my experience the most important thing is to completely separate the "test driver" code from the "page-wrapping code". By "test driver" I mean the actual test logic (NUnit or similar) which might have code that looks like:

...

var wrapper = new SearchPageWrapper(browser);
wrapper.ClickAdvancedSearch();
wrapper.EnterSearchPhrase("dog");
wrapper.SetSortBy(SortType.Date);
wrapper.ExecuteSearch();

... the page-wrapper exposes this logical interface to the page itself while hiding the implementation details (which can be complex and change often with the page layout):

public class SearchPageWrapper {
  ...
  public void ClickAdvancedSearch() {
    _browser.Buttons("advSearch").Click();
  }

  public void EnterSearchPhrase(string phrase) {
    _browser.TextBox(Find.ByName("phrase")).TypeText(phrase);
  }
  ... etc ...
}

This way you build a library of wrappers that makes it easy to add new test cases, and also makes it less painful to deal with layout changes that affect the way that Watin finds & interacts with the elements on the pages.

Answer 4

The below link is documentation on how to get WebAii working with all the popular CI servers (CC, TeamCity and TFS Build)

http://www.artoftest.com/support/webaii/topicsindex.aspx?topic=cioverview