Visual Studio 2013: Creating ordered tests

Question

Can someone suggest a way to run tests in a specific order in Visual Studio 2013 Express? Is there a way to create a playlist for the tests, which also defines the order in which to run them?

By the way: These are functional tests, using Selenium, written as unit tests in C#/Visual Studio. Not actual unit tests. Sometimes a regression test suite is so big it takes a while to run through all the tests. In these cases, I've often seen the need to run the test in a prioritized order. Or, there can be cases where it's difficult to run some tests without some other tests having been run before. In this regard, it's a bit more complicated than straight unit tests (which is the reason why it's normally done by test professionals, while unit tests are done by developers).

I've organised the tests in classes with related test methods. Ex.: All login tests are in a class called LoginTests, etc.

Class LoginTests:
- AdminCanLogin (...)
- UserCanLogin (...)
- IncorrectLoginFails (...)
- ...

CreatePostTests
- CanCreateEmptyPost (...)
- CanCreateBasicPost (...)
...

These classes are unit test classes, in their own project. They in turn calls classes and methods in a class library that uses Selenium.

MS suggests creating an "Ordered Unit Test" project. However, this is not available in the Express edition.

Answer 1

To address your playlist request directly see MS article: http://msdn.microsoft.com/en-us/library/hh270865.aspx Resharper also has a nice test play list tool as well.

Here is an article on how to setup Ordered Tests but you cannot use this feature with Express as it requires Visual Studio Ultimate, Visual Studio Premium, Visual Studio Test Professional. http://msdn.microsoft.com/en-us/library/ms182631.aspx

If you need them ordered then they are more then likely integration tests. I am assuming you would like them ordered so you can either prepare data for the test or tear data back down after the test.

There are several ways to accommodate this requirement if it is the case. Using MSTest there are 4 attributes for this you can see more details of when they are executed here http://blogs.msdn.com/b/nnaderi/archive/2007/02/17/explaining-execution-order.aspx.

My other suggestion would be to have a helper class to preform the tasks(not tests) you are looking to have done in order, to be clear this class would not be a test class just a normal class with common functionality that would be called from within your tests.

If you need a test to create a product so another test can use that product and test that it can be added to a shopping cart then I would create a "SetupProduct" method that would do this for you as I am sure you would be testing various things that would require a product. This would prevent you from having test dependencies.

With that said, integration tests are good to verify end to end processes but where possible and applicable it might be easier to mock some or all dependencies such as your repositories. I use the Moq framework and find it really easy to work with.

This code is from the blog post linked above, I am placing it here in case the link ever dies.

Here is an example of a test class using the setup / tear down attributes to help with your tests.

[TestClass]
public class VSTSClass1
{

    private TestContext testContextInstance;

    public TestContext TestContext
    {
        get
        {
            return testContextInstance;
        }
        set
        {
            testContextInstance = value;
        }
    }

    [ClassInitialize]
    public static void ClassSetup(TestContext a)
    {
        Console.WriteLine("Class Setup");
    }

    [TestInitialize]
    public void TestInit()
    {
        Console.WriteLine("Test Init");

    }

    [TestMethod]
    public void Test1()
    {
        Console.WriteLine("Test1");
    }

    [TestMethod]
    public void Test2()
    {
        Console.WriteLine("Test2");

    }

    [TestMethod]
    public void Test3()
    {
        Console.WriteLine("Test3");
    }

    [TestCleanup]
    public void TestCleanUp()
    {
        Console.WriteLine("TestCleanUp");
    }

    [ClassCleanup]
    public static void ClassCleanUp()
    {
        Console.WriteLine("ClassCleanUp");
    }
}

Here is the order that the methods were fired.

Class Setup
Test Init
Test1
TestCleanUp
Test Init
Test2
TestCleanUp
Test Init
Test3
TestCleanUp
ClassCleanUp

If you give more information on what you are trying to accomplish I would be happy to assist you in when to use which attribute or when to use the help class, note the helper class is NOT a test class just a standard class that has methods you can utilize to do common tasks that may be needed for multiple tests.