NUnit has a feature called Values, like below:
[Test]
public void MyTest(
[Values(1,2,3)] int x,
[Values("A","B")] string s)
{
// ...
}
This means that the test method will run six times:
MyTest(1, "A")
MyTest(1, "B")
MyTest(2, "A")
MyTest(2, "B")
MyTest(3, "A")
MyTest(3, "B")
We're using MSTest now, but is there any equivalent for this so that I can run the same test with multiple parameters?
[TestMethod]
public void Mytest()
{
// ...
}
On
MSTest does not support that feature, but you can implement your own attribute to achieve that.
Have a look at Enabling parameterized tests in MSTest using PostSharp.
On
Not exactly the same as NUnit's Value (or TestCase) attributes, but MSTest has the DataSource attribute, which allows you to do a similar thing.
You can hook it up to database or XML file - it is not as straightforward as NUnit's feature, but it does the job.
On
MSTest has a powerful attribute called DataSource. Using this you can perform data-driven tests as you asked. You can have your test data in XML, CSV, or in a database. Here are few links that will guide you
On
Here is a reimplentation of NUnits [Combinatorial], [Sequential] and [Values].
Unlike NUnit, which assumes [Combinatorial] by default, in MSTest we must always specify which one we want, else the [Values] attribute will not have any effect.
Usage:
[TestClass]
public class TestMethods
{
[TestMethod, Combinatorial]
public void EnumIterationTestMethod(Season season) => Console.WriteLine(season);
[TestMethod, Combinatorial]
public void BoolIterationTestMethod(bool boolean) => Console.WriteLine(boolean);
[TestMethod, Combinatorial]
public void CombinatoralValuesIterationTestMethod(Season season, bool boolean) => Console.WriteLine($"{season} {boolean}");
[TestMethod, Sequential]
public void SequentialCombinatoralIterationTestMethod(
[Values(1, 2, 3)] int param1,
[Values("A", "B")] string param2) => Console.WriteLine($"{param1} {param2 ?? "null"}");
[TestMethod, Combinatorial]
public void CombinatoralIterationTestMethod(
[Values(1, 2, 3)] int param1,
[Values("A", "B")] string param2) => Console.WriteLine($"{param1} {param2 ?? "null"}");
}
Code:
[AttributeUsage(AttributeTargets.Parameter, AllowMultiple = false)]
public class ValuesAttribute : Attribute
{
public object?[] Values { get; }
public ValuesAttribute(params object?[] values)
{
Values = values;
}
}
[AttributeUsage(AttributeTargets.Method, AllowMultiple = false)]
public class CombinatorialAttribute : Attribute, ITestDataSource
{
public IEnumerable<object?[]> GetData(MethodInfo methodInfo)
{
var values = Utils.GetPossibleValuesForEachParameter(methodInfo);
return Utils.CreateCombinations(values);
}
public string? GetDisplayName(MethodInfo methodInfo, object?[]? data)
{
if (data != null)
{
return $"{methodInfo.Name} ({string.Join(", ", data.Select(e => e ?? "null"))})";
}
return null;
}
}
[AttributeUsage(AttributeTargets.Method, AllowMultiple = false)]
public class SequentialAttribute : Attribute, ITestDataSource
{
public IEnumerable<object?[]> GetData(MethodInfo methodInfo)
{
var values = Utils.GetPossibleValuesForEachParameter(methodInfo);
return Utils.ZipLongestFillWithNull(values);
}
public string? GetDisplayName(MethodInfo methodInfo, object?[]? data)
{
if (data != null)
{
return $"{methodInfo.Name} ({string.Join(", ", data.Select(e => e ?? "null"))})";
}
return null;
}
}
public static class Utils
{
public static List<List<object?>> GetPossibleValuesForEachParameter(MethodInfo methodInfo)
{
List<List<object?>> values = new();
foreach (var parameter in methodInfo.GetParameters())
{
var attribute = parameter.GetCustomAttribute<ValuesAttribute>();
if (attribute == null || attribute.Values.Length == 0)
{
if (parameter.ParameterType.IsEnum)
{
values.Add(Enum.GetValues(parameter.ParameterType).Cast<object?>().ToList());
continue;
}
if (parameter.ParameterType == typeof(bool))
{
values.Add(new List<object?> { true, false });
continue;
}
if (attribute == null)
{
throw new InvalidOperationException($"{parameter.Name} should have a [Values(...)] attribute set");
}
else
{
throw new InvalidOperationException($"[Values] {parameter.ParameterType} {parameter.Name} is only valid for Enum or Boolean types. Consider using the attribute constructor [Values(...)].");
}
}
values.Add(attribute.Values.ToList());
}
return values;
}
public static IEnumerable<object?[]> ZipLongestFillWithNull(List<List<object?>> values)
{
var longest = values.Max(e => e.Count);
foreach (var list in values)
{
if (list.Count < longest)
{
var diff = longest - list.Count;
list.AddRange(Enumerable.Repeat<object?>(null, diff));
}
}
for (int i = 0; i < longest; i++)
{
yield return values.Select(e => e[i]).ToArray();
}
}
public static IEnumerable<object?[]> CreateCombinations(List<List<object?>> values)
{
var indices = new int[values.Count];
while (true)
{
// Create new arguments
var arg = new object?[indices.Length];
for (int i = 0; i < indices.Length; i++)
{
arg[i] = values[i][indices[i]];
}
yield return arg!;
// Increment indices
for (int i = indices.Length - 1; i >= 0; i--)
{
indices[i]++;
if (indices[i] >= values[i].Count)
{
indices[i] = 0;
if (i == 0)
yield break;
}
else
break;
}
}
}
}
This feature is in pre-release now and works with Visual Studio 2015.
For example:
[TestClass]
public class UnitTest1
{
[TestMethod]
[DataRow(1, 2, 2)]
[DataRow(2, 3, 5)]
[DataRow(3, 5, 8)]
public void AdditionTest(int a, int b, int result)
{
Assert.AreEqual(result, a + b);
}
}
There is, of course, another way to do this which has not been discussed in this thread, i.e. by way of inheritance of the class containing the TestMethod. In the following example, only one TestMethod has been defined but two test cases have been made.
In Visual Studio 2012, it creates two tests in the TestExplorer:
DemoTest_A12_B4.test
public class Demo
{
int a, b;
public Demo(int _a, int _b)
{
this.a = _a;
this.b = _b;
}
public int Sum()
{
return this.a + this.b;
}
}
public abstract class DemoTestBase
{
Demo objUnderTest;
int expectedSum;
public DemoTestBase(int _a, int _b, int _expectedSum)
{
objUnderTest = new Demo(_a, _b);
this.expectedSum = _expectedSum;
}
[TestMethod]
public void test()
{
Assert.AreEqual(this.expectedSum, this.objUnderTest.Sum());
}
}
[TestClass]
public class DemoTest_A12_B4 : DemoTestBase
{
public DemoTest_A12_B4() : base(12, 4, 16) { }
}
public abstract class DemoTest_B10_Base : DemoTestBase
{
public DemoTest_B10_Base(int _a) : base(_a, 10, _a + 10) { }
}
[TestClass]
public class DemoTest_B10_A5 : DemoTest_B10_Base
{
public DemoTest_B10_A5() : base(5) { }
}
EDIT 4: Looks like this is completed in MSTest V2 June 17, 2016: https://blogs.msdn.microsoft.com/visualstudioalm/2016/06/17/taking-the-mstest-framework-forward-with-mstest-v2/
Original Answer:
As of about a week ago in Visual Studio 2012 Update 1 something similar is now possible:
[DataTestMethod]
[DataRow(12,3,4)]
[DataRow(12,2,6)]
[DataRow(12,4,3)]
public void DivideTest(int n, int d, int q)
{
Assert.AreEqual( q, n / d );
}
EDIT: It appears this is only available within the unit testing project for WinRT/Metro. Bummer
EDIT 2: The following is the metadata found using "Go To Definition" within Visual Studio:
#region Assembly Microsoft.VisualStudio.TestPlatform.UnitTestFramework.dll, v11.0.0.0
// C:\Program Files (x86)\Microsoft SDKs\Windows\v8.0\ExtensionSDKs\MSTestFramework\11.0\References\CommonConfiguration\neutral\Microsoft.VisualStudio.TestPlatform.UnitTestFramework.dll
#endregion
using System;
namespace Microsoft.VisualStudio.TestPlatform.UnitTestFramework
{
[AttributeUsage(AttributeTargets.Method, AllowMultiple = false)]
public class DataTestMethodAttribute : TestMethodAttribute
{
public DataTestMethodAttribute();
public override TestResult[] Execute(ITestMethod testMethod);
}
}
EDIT 3: This issue was brought up in Visual Studio's UserVoice forums. Last Update states:
STARTED · Visual Studio Team ADMIN Visual Studio Team (Product Team, Microsoft Visual Studio) responded · April 25, 2016 Thank you for the feedback. We have started working on this.
Pratap Lakshman Visual Studio
On
I couldn't get The DataRowAttribute to work in Visual Studio 2015, and this is what I ended up with:
[TestClass]
public class Tests
{
private Foo _toTest;
[TestInitialize]
public void Setup()
{
this._toTest = new Foo();
}
[TestMethod]
public void ATest()
{
this.Perform_ATest(1, 1, 2);
this.Setup();
this.Perform_ATest(100, 200, 300);
this.Setup();
this.Perform_ATest(817001, 212, 817213);
this.Setup();
}
private void Perform_ATest(int a, int b, int expected)
{
// Obviously this would be way more complex...
Assert.IsTrue(this._toTest.Add(a,b) == expected);
}
}
public class Foo
{
public int Add(int a, int b)
{
return a + b;
}
}
The real solution here is to just use NUnit (unless you're stuck in MSTest like I am in this particular instance).
On
The OP example was about an NUnit feature that easily allows to have a cartesian product of provided values. As far as I was able to tell, no answer here covered that part. I saw this as a little challenge and ended up with the following implementation.
[Edit: Array based refactoring + Zip values]
I did some refactoring to the original Enumerator based version (see post history) to now use only Arrays and loop throuch indices instead. I also took the opporunity to add a new Zip type of values that will match a different value to every set created by the cartesian produce. This may be useful to add an ExpectedResult for instance.
It is still not really optimized so feel free to suggest improvements.
#nullable enable
public enum ValuesType
{
Undefined = 0,
Cartesian = 1,
/// <summary>
/// Values will be <see cref="Enumerable.Zip{TFirst, TSecond, TResult}(IEnumerable{TFirst}, IEnumerable{TSecond}, Func{TFirst, TSecond, TResult})">Zipped</see> with the cartesian produce of the other parameters.
/// </summary>
Zip = 2
}
[AttributeUsage(AttributeTargets.Parameter, AllowMultiple = false)]
public class ValuesAttribute : Attribute
{
public ValuesType ValuesType { get; }
public object[] Values { get; }
public ValuesAttribute(params object[] values)
: this(ValuesType.Cartesian, values)
{ }
public ValuesAttribute(ValuesType valuesType, params object[] values)
{
ValuesType = valuesType;
Values = values;
}
}
[AttributeUsage(AttributeTargets.Method, AllowMultiple = false)]
public class ValuesDataSourceAttribute : Attribute, ITestDataSource
{
public IEnumerable<object[]> GetData(MethodInfo methodInfo)
{
var parameters = methodInfo.GetParameters();
var values = new (ValuesType Type, object[] Values, int Index)[parameters.Length];
for(var i=0; i < parameters.Length; i++)
{
var parameter = parameters[i];
var attribute = parameter.GetCustomAttribute<ValuesAttribute>();
if (attribute != null)
{
if (attribute.Values.Any(v => !parameter.ParameterType.IsAssignableFrom(v.GetType())))
throw new InvalidOperationException($"All values of {nameof(ValuesAttribute)} must be of type {parameter.ParameterType.Name}. ParameterName: {parameter.Name}.");
switch (attribute.ValuesType)
{
case ValuesType.Cartesian:
values[i] = (ValuesType.Cartesian, attribute.Values, 0);
break;
case ValuesType.Zip:
values[i] = (ValuesType.Zip, attribute.Values, 0);
break;
}
}
else if (parameter.ParameterType == typeof(bool))
values[i] = (ValuesType.Cartesian, new object[] { false, true }, 0);
else if (parameter.ParameterType.IsEnum)
values[i] = (ValuesType.Cartesian, Enum.GetValues(parameter.ParameterType).Cast<Object>().ToArray(), 0);
else
throw new InvalidOperationException($"All parameters must have either {nameof(ValuesAttribute)} attached or be a bool or an Enum . ParameterName: {parameter.Name}.");
}
//Since we are using ValueTuples, it is essential that once we created our collection, we stick to it. If we were to create a new one, we would end up with a copy of the ValueTuples that won't be synced anymore.
var cartesianTotalCount = values.Where(v => v.Type == ValuesType.Cartesian).Aggregate(1, (actualTotal, currentValues) => actualTotal * currentValues.Values.Length);
if (values.Any(v => v.Type == ValuesType.Zip && v.Values.Length != cartesianTotalCount))
throw new InvalidOperationException($"{nameof(ValuesType.Zip)} typed attributes must have as many values as the produce of all the others ({cartesianTotalCount}).");
bool doIncrement;
for(var globalPosition = 0; globalPosition < cartesianTotalCount; globalPosition++)
{
yield return values.Select(v => v.Values[v.Index]).ToArray();
doIncrement = true;
for (var i = values.Length - 1; i >= 0 && doIncrement; i--)
{
switch (values[i].Type)
{
case ValuesType.Zip:
values[i].Index++;
break;
case ValuesType.Cartesian:
if (doIncrement && ++values[i].Index >= values[i].Values.Length)
values[i].Index = 0;
else
doIncrement = false;
break;
default:
throw new InvalidOperationException($"{values[i].Type} is not supported.");
}
}
}
}
public string GetDisplayName(MethodInfo methodInfo, object[] data)
{
return data.JoinStrings(" / ");
}
}
Usage:
[TestMethod]
[ValuesDataSource]
public void Test([Values("a1", "a2")] string a, [Values(1, 2)] int b, bool c, System.ConsoleModifiers d, [Values(ValuesType.Zip, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24)] int asserts)
{
//Arrange / Act / Assert
//Cases would be
// a1, 1, false, System.ConsoleModifiers.Alt, 1
// a1, 1, false, System.ConsoleModifiers.Shift, 2
// a1, 1, false, System.ConsoleModifiers.Control, 3
// a1, 1, true, System.ConsoleModifiers.Alt, 4
// a1, 1, true, System.ConsoleModifiers.Shift, 5
// a1, 1, true, System.ConsoleModifiers.Control, 6
// a1, 2, false, System.ConsoleModifiers.Alt, 7
// a1, 2, false, System.ConsoleModifiers.Shift, 8
// a1, 2, false, System.ConsoleModifiers.Control, 9
// a1, 2, true, System.ConsoleModifiers.Alt, 10
// a1, 2, true, System.ConsoleModifiers.Shift, 11
// a1, 2, true, System.ConsoleModifiers.Control, 12
// a2, 1, false, System.ConsoleModifiers.Alt, 13
// a2, 1, false, System.ConsoleModifiers.Shift, 14
// a2, 1, false, System.ConsoleModifiers.Control, 15
// a2, 1, true, System.ConsoleModifiers.Alt, 16
// a2, 1, true, System.ConsoleModifiers.Shift, 17
// a2, 1, true, System.ConsoleModifiers.Control, 18
// a2, 2, false, System.ConsoleModifiers.Alt, 19
// a2, 2, false, System.ConsoleModifiers.Shift, 20
// a2, 2, false, System.ConsoleModifiers.Control, 21
// a2, 2, true, System.ConsoleModifiers.Alt, 22
// a2, 2, true, System.ConsoleModifiers.Shift, 23
// a2, 2, true, System.ConsoleModifiers.Control, 24
}
It's very simple to implement - you should use TestContext property and TestPropertyAttribute.
public TestContext TestContext { get; set; }
private List<string> GetProperties()
{
return TestContext.Properties
.Cast<KeyValuePair<string, object>>()
.Where(_ => _.Key.StartsWith("par"))
.Select(_ => _.Value as string)
.ToList();
}
//usage
[TestMethod]
[TestProperty("par1", "http://getbootstrap.com/components/")]
[TestProperty("par2", "http://www.wsj.com/europe")]
public void SomeTest()
{
var pars = GetProperties();
//...
}
I prepared few extension methods to simplify access to the TestContext property and act like we have several test cases. See example with processing simple test properties here:
[TestMethod]
[TestProperty("fileName1", @".\test_file1")]
[TestProperty("fileName2", @".\test_file2")]
[TestProperty("fileName3", @".\test_file3")]
public void TestMethod3()
{
TestContext.GetMany<string>("fileName").ForEach(fileName =>
{
//Arrange
var f = new FileInfo(fileName);
//Act
var isExists = f.Exists;
//Asssert
Assert.IsFalse(isExists);
});
}
and example with creating complex test objects:
[TestMethod]
//Case 1
[TestProperty(nameof(FileDescriptor.FileVersionId), "673C9C2D-A29E-4ACC-90D4-67C52FBA84E4")]
//...
public void TestMethod2()
{
//Arrange
TestContext.For<FileDescriptor>().Fill(fi => fi.FileVersionId).Fill(fi => fi.Extension).Fill(fi => fi.Name).Fill(fi => fi.CreatedOn, new CultureInfo("en-US", false)).Fill(fi => fi.AccessPolicy)
.ForEach(fileInfo =>
{
//Act
var fileInfoString = fileInfo.ToString();
//Assert
Assert.AreEqual($"Id: {fileInfo.FileVersionId}; Ext: {fileInfo.Extension}; Name: {fileInfo.Name}; Created: {fileInfo.CreatedOn}; AccessPolicy: {fileInfo.AccessPolicy};", fileInfoString);
});
}
Take a look to the extension methods and set of samples for more details.
It is unfortunately not supported in older versions of MSTest. Apparently there is an extensibility model and you can implement it yourself. Another option would be to use data-driven tests.
My personal opinion would be to just stick with NUnit though...
As of Visual Studio 2012, update 1, MSTest has a similar feature. See McAden's answer.