Error implementing C# .NET 7 RestAPI RabbitMQ - MassTransit Dead Letter Queue (DLQ)

Question

I'm trying to get better at using micro services and I'm stuck on something. I'm working with:

• ASP.NET Core 7 Web API (for the micro services)
• C#
• SQL Server
• RabbitMQ and MassTransit
• Docker compose (using Linux Containers)

I'm still new and learning as I go. So, I've got a basic queue working, but I'm having trouble when things go wrong, especially with managing Dead Letters from the calling API.

Here's what's happening: something sends a POST request and the queue says "Cool, got it" (aka "Accepted").

Now, what I want is: If I force an error, I want the message to try to resend a few times (retry) and then, if it still can’t send, move to the error Dead Letter Queue (DLQ).

But here's the weird part: when I send a request, it goes straight to the DLQ Error Queue first, then jumps back to the main queue, tries to send a few times (retries), and if it still doesn’t work, hops back to the error queue.

The code can be found on my GitHub Repo here. It's a POC which uses docker containers.

This diagram should illustrate what I am hoping to achieve:

My Employee API DI for RabbitMQ and MassTransit:

builder.Services.AddMassTransit(config =>
{
     config.AddConsumer<EmployeeDtoEventConsumer>();

     config.UsingRabbitMq((ctx, cfg) =>
     {
         cfg.Host(builder.Configuration["EventBusSettings:HostAddress"]);

         cfg.ReceiveEndpoint("employee_q", c =>
         {
             c.PrefetchCount = 16;

             c.ConfigureConsumer<EmployeeDtoEventConsumer>(ctx);

         });

         cfg.UseMessageRetry(retryConfigurator =>
         {
             retryConfigurator.Interval(3, TimeSpan.FromSeconds(1));
             retryConfigurator.Handle<Exception>();
         });
     });
});

builder.Services.AddScoped<EmployeeDtoEventConsumer>();

And this is the DQLConsumer API DI:

builder.Services.AddMassTransit(config =>
{
     config.AddConsumer<DlqConsumer>();

     config.UsingRabbitMq((ctx, cfg) =>
     {
         cfg.Host(builder.Configuration["EventBusSettings:HostAddress"]);

         cfg.ReceiveEndpoint("employee_q_error", c =>
         {
             c.PrefetchCount = 16;

             c.ConfigureConsumer<DlqConsumer>(ctx);

         });
     });
});

builder.Services.AddScoped<DlqConsumer>();

Can anyone help me figure out:

1. How to make the message go to the DL error queue only after it's done retrying in the main queue?
2. Any easy tips or tricks on managing dead letters in RabbitMQ and MassTransit?

Thanks a bunch for any help you can give!

Answer 1

Order matters, or you can just simplify:

builder.Services.AddMassTransit(config =>
{
     config.AddConsumer<EmployeeDtoEventConsumer>();

     config.UsingRabbitMq((ctx, cfg) =>
     {
         cfg.Host(builder.Configuration["EventBusSettings:HostAddress"]);

         cfg.ReceiveEndpoint("employee_q", c =>
         {
             c.PrefetchCount = 16;

             c.UseMessageRetry(retryConfigurator =>
             {
                 retryConfigurator.Interval(3, TimeSpan.FromSeconds(1));
                 retryConfigurator.Handle<Exception>();
             });

             c.ConfigureConsumer<EmployeeDtoEventConsumer>(ctx);
         });
     });
});

Also, REMOVE this entirely:

builder.Services.AddScoped<EmployeeDtoEventConsumer>();