How can I write an integration test for Marten (on top of PostgreSQL) that includes asynchronous projections in an asp.net context?

Question

I'm looking for a way to test functionality written using Marten. Not just as a unit test, but where integration with asp.net API's and actual saving to the database is tested. Moreover, I want to test whether async projections are generated as well. So to sum up, I would like to test whether the following is executed correctly:

1. Domain Driven Design styled command is received by a controller action
2. The command is translated into a domain event and that event has been saved to PostgreSQL
3. After a while a projection is asynchronously generated and saved to PostgreSQL as well

Marten does not provide concrete examples in their documentation. They do have unit-test in their codebase, but these are not standalone enough to provide enough information to build your own integration tests in my opinion.

Given the following setup (inspired by Marten examples):

Command

public record StartQuest(
    Guid Id, 
    string Name, 
    int Day, 
    string Location, 
    params string[] Members) : IRequest;

Aggregate, projection and events

public class Quest
{
    public Guid Id { get; set; }
}

public class QuestPartyProjection : SingleStreamAggregation<QuestParty>
{
    public QuestParty Create(QuestStarted @event) => new() { Name = @event.Name };
    public void Apply(QuestParty view, MembersJoined @event) => view.Members.Fill(@event.Members);
}

public class QuestParty
{
    public Guid Id { get; set; }
    public List<string> Members { get; set; } = new();
    public string Name { get; set; }
}

public record MembersJoined(int Day, string Location, string[] Members);

public record QuestStarted(string Name);

MediatR is used to delegate commands to command handlers

[ApiController]
[Route("[controller]")]
public class QuestController : ControllerBase
{
    private readonly IMediator mediator;

    public QuestController(IMediator mediator)
    {
        this.mediator = mediator;
    }

    [HttpPost("[Action]")]
    public async Task<IActionResult> Start([FromBody] StartQuest command, CancellationToken cancellationToken)
    {
        await mediator.Send(command, cancellationToken);
        return Ok();
    }
}

internal sealed class StartHandler : IRequestHandler<StartQuest>
{
    private readonly IDocumentSession session;

    public StartHandler(IDocumentSession session)
    {
        this.session = session;
    }

    public async Task<Unit> Handle(StartQuest command, CancellationToken cancellationToken)
    {
        var started = new QuestStarted(command.Name);
        var joined1 = new MembersJoined(command.Day, command.Location, command.Members);

        session.Events.StartStream(typeof(Quest), command.Id, started, joined1);
        await session.SaveChangesAsync(cancellationToken);
        return Unit.Value;
    }
}

Which involves the following setup in my .net application:

builder.Services.AddMarten(x =>
{
    x.Connection(builder.Configuration.GetConnectionString("Marten")!);
    x.Projections.Add<QuestPartyProjection>(ProjectionLifecycle.Async);
})
    .OptimizeArtifactWorkflow(TypeLoadMode.Static)
    .AddAsyncDaemon(DaemonMode.HotCold)
    .UseLightweightSessions()
    .InitializeWith();;
builder.Services.AddMediatR(typeof(StartHandler));

Answer 1

UPDATE 2023-12-10: I wrote documentation for Marten which gets updated and works better here: Integration testing - Marten

I created integration teest using xunit (for declaring and executing tests), TestContainers (for running a Docker PostgreSQL container o,n the fly), and Microsoft.AspNetCore.Mvc.Testing for bootstrapping the application in memory for functional end-to-end tests.

My csproj has these package references (and a reference to the project to be tested):

<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.4.0" />
<PackageReference Include="xunit" Version="2.4.2" />
<PackageReference Include="xunit.runner.visualstudio" Version="2.4.5">
    <PrivateAssets>all</PrivateAssets>
    <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
<PackageReference Include="Testcontainers" Version="2.2.0" />
<PackageReference Include="Microsoft.AspNetCore.Mvc.Testing" Version="7.0.0" />

First, since mvc.testing need a public entrypoint, I've added this to the program.cs

public partial class Program { } // Expose the Program class for use with WebApplicationFactory<T>

Now in the following code a custom WebApplicationFactory is created on top of Microsoft's end to end testing framework. A new docker instance for PostgreSQL is created for every run and the application configuration is overwritten by the connectionstring that points to that PostgreSQL instance.

public class CustomWebApplicationFactory<TProgram>
    : WebApplicationFactory<TProgram> where TProgram : class
{
    public TestcontainerDatabase Testcontainers { get; } = 
        new TestcontainersBuilder<PostgreSqlTestcontainer>()
        .WithDatabase(new PostgreSqlTestcontainerConfiguration
        {
            Database = "db",
            Username = "postgres",
            Password = "postgres",
        })
        .Build();

    public override async ValueTask DisposeAsync()
    {
        await Testcontainers.StopAsync();
        await base.DisposeAsync();
    }

    protected override IHost CreateHost(IHostBuilder builder)
    {
        Testcontainers.StartAsync().GetAwaiter().GetResult();
        builder.ConfigureHostConfiguration(configBuilder =>
        {
            configBuilder.AddInMemoryCollection(
                new Dictionary<string, string>
                {
                    ["ConnectionStrings:Marten"] = Testcontainers.ConnectionString
                }!);
        });
        return base.CreateHost(builder);
    }
}

I created an abstract integration class for reuse:

public abstract class IntegrationTest : IClassFixture<CustomWebApplicationFactory<Program>>
{
    protected readonly CustomWebApplicationFactory<Program> Factory;
    protected IDocumentStore DocumentStore => Factory.Services.GetRequiredService<IDocumentStore>();

    protected IntegrationTest(CustomWebApplicationFactory<Program> factory)
    {
        Factory = factory;
    }

    /// <summary>
    /// 1. Start generation of projections
    /// 2. Wait for projections to be projected
    /// </summary>
    protected async Task GenerateProjectionsAsync()
    {
        using var daemon = await DocumentStore.BuildProjectionDaemonAsync();
        await daemon.StartAllShards();
        await daemon.WaitForNonStaleData(5.Seconds());
    }

    protected IDocumentSession OpenSession() => DocumentStore.LightweightSession();
}

And now my actual unit test is pretty straightforward:

1. Create a command to start the quest
2. Send it to my running asp.net application using a ready-to-use httpclient. JSON serialization is done by build in "System.Net.Http.Json"
3. Await asynchronous generation of projections
4. Fetch the projection from the PostgreSQL database and validate the content

public class QuestPartyTests : IntegrationTest
{
    public QuestPartyTests(CustomWebApplicationFactory<Program> factory) : base(factory)
    {
    }

    [Fact]
    public async Task Start_ShouldResultIn_Party()
    {
        // Arrange
        var command = new StartQuest(
            Guid.NewGuid(),
            Name: "Destroy the One Ring",
            Day: 1,
            Location: "Hobbiton",
            Members: new []{ "Frodo", "Sam"});

        // Act
        await Factory.CreateClient().PostAsync("quest/start", JsonContent.Create(command));

        await GenerateProjectionsAsync();

        // Assert
        await using var session = OpenSession();
        var party = await session.Query<QuestParty>().SingleAsync();

        Assert.Equal(command.Name, party.Name);
        Assert.Equal(command.Members, party.Members);
    }
}