AutoFixture Tests Stop Showing Up in Test Runner

Question

AutoFixture (or my misuse of it) seems to have caused the xunit test runner to stop showing individual tests in the tree view for each instance of inline data. Usually if I use [Theory] and [InlineData] I get individual tests showing up in the test runner tree view, one for each [InlineData]. That no longer happens after the following:

I created a custom AutoMoqDataAttribute class to always use the AutoMoqCustomization and disable recursion:

public sealed class AutoMoqDataAttribute : AutoDataAttribute
{
    /// <summary>
    /// Automaticaly adds the "Auto Moq" customization to fixures that use the
    /// AutoDataAttribute attribute.
    /// </summary>
    public AutoMoqDataAttribute() : base(() =>
    {
        // Create a fixture we can customize.
        var fixture = new Fixture();

        // Remove the recursion checks as we have objects that require recursion.
        var throwingRecursionBehavior = fixture.Behaviors.FirstOrDefault(behavior => behavior.GetType() == typeof(ThrowingRecursionBehavior));
        if (throwingRecursionBehavior != null) fixture.Behaviors.Remove(throwingRecursionBehavior);

        // Have to also add in the OmitOnRecursionBehavior to avoid a stack overflow error
        // presumably, since we removed the above, and without this, AutoFixture follows every
        // object and just keeps trying to auto fill its data. By following the circular
        // references it just goes on forever.
        fixture.Behaviors.Add(new OmitOnRecursionBehavior());

        // Use the "Auto Moq" customization so we have AutoFixture automatically mock objects for us.
        fixture.Customize(new AutoMoqCustomization());

        return fixture;
    }) { }
}

Then, I want this to work when I use Theory/InlineData, so I also create this:

    public class InlineAutoMoqDataAttribute : InlineAutoDataAttribute
    {
        public InlineAutoMoqDataAttribute(params object[] values) : base(new AutoMoqDataAttribute(), values) { }
    }

Finally, I try to run some tests:

    public class TestClass
    {
        public bool Val { get; set; }
    }

    [Theory]
    [InlineData("blah 1", true)]
    [InlineData("blah 2", true)]
    public void Test1(string dummy, bool expected)
    {
        Assert.Equal(expected, true);
    }

The above works fine... but then I run this:

    [Theory]
    [InlineAutoMoqData("blah 1", false)]
    [InlineAutoMoqData("blah 2", false)]
    public void Test2(string dummy, bool expected, TestClass sut)
    {
        Assert.Equal(expected, sut.Val);
    }

The tests run but I don't see two tests, one for each InlineAutoMoqData, in test runner as I usually do. I can see them if I click the test and look in the "Test Detail Summary" panel, which would almost be good enough except I can't re-run failed tests (and I mean tests for individual InlineData cases) from there. I can from the tree view but again, they're not showing up there so that is a big problem as it runs all InlineAutoMoqData tests and I want to isolate and just run e.g. one that is failing.

If I add this attribute:

[DataDiscoverer("Xunit.Sdk.InlineDataDiscoverer", "xunit.core")]

to my InlineAutoMoqDataAttribute, the individual tests, one for each InlineAutoMoqData, show up in test explorer but then I get this error:

"InvalidOperationException : The test method expected 2 parameter values, but 1 parameter value was provided."

meaning AutoFixture auto mock values don't work.

Is there any way to have my custom InlineAutoMoqData attribute and still have tests for each individual InlineAutoMoqData showing up in test explorer? Is this a bug I should report to AutoFixture, or am I doing something wrong?

Answer 1

This is a known issue in AutoFixture, though I'm not sure there is a formal issue opened in the GitHub repository. If there isn't one you are welcome to create an issue and track the progress on it there.