I'm using Hangfire and for some reason, two server instances are created during startup.
During startup I use
services.AddHangfire((provider, hangfireConfig) => hangfireConfig
.UseDatabase(storageSettings.StorageProvider, storageSettings.ConnectionString, config)
.UseConsole());
services.AddHangfireServer(options => config.GetSection("HangfireSettings:Server").Bind(options));
Later I build
var app = builder.Build();
And finally add the dashboard
var dashboardOptions = config.GetSection("HangfireSettings:Dashboard").Get<DashboardOptions>();
dashboardOptions.Authorization = new[]
{
new HangfireCustomBasicAuthenticationFilter
{
User = config.GetSection("HangfireSettings:Credentials:User").Value,
Pass = config.GetSection("HangfireSettings:Credentials:Password").Value
}
};
app.UseHangfireDashboard(config["HangfireSettings:Route"], dashboardOptions);
Once I run the app (app.Run()) two server instances are created.
From the debug log I cannot see server with guid C7518906-D440-4113-85AD-0C1A97E01ACC, but the heartbeat is updated a few seconds before the instance shown in the log.
[09:01:20 INF] Starting Hangfire Server using job storage: 'SQL Server: xxxx'
[09:01:20 INF] Using the following options for SQL Server job storage: Queue poll interval: 00:00:01.
[09:01:20 INF] Using the following options for Hangfire Server:
Worker count: 5
Listening queues: 'default', 'default', 'notdefault', 'background', 'export', 'provisioning'
Shutdown timeout: 00:00:15
Schedule polling interval: 00:00:15
[09:01:20 DBG] Execution loop BackgroundServerProcess:6446fe1e has started in 14,3531 ms
[09:01:20 INF] Application started. Press Ctrl+C to shut down.
[09:01:20 INF] Hosting environment: NotDevelopment
[09:01:20 INF] Server 55b4c516-d5a9-4516-be31-29b9a4e83f1c successfully announced in 189,132 ms
[09:01:20 INF] Server 55b4c516-d5a9-4516-be31-29b9a4e83f1c is starting the registered dispatchers: ServerWatchdog, ServerJobCancellationWatcher, ExpirationManager, CountersAggregator, SqlServerHeartbeatProcess, Worker, DelayedJobScheduler, RecurringJobScheduler...
[09:01:20 DBG] Execution loop ServerHeartbeatProcess:bab9cda3 has started in 7,4454 ms
[09:01:20 DBG] Execution loop ServerWatchdog:d2946b36 has started in 7,503 ms
[09:01:20 DBG] Execution loop ServerJobCancellationWatcher:8325617f has started in 11,7685 ms
[09:01:20 DBG] Execution loop ExpirationManager:e95adc5c has started in 10,9894 ms
[09:01:20 DBG] Execution loop CountersAggregator:cedbdca3 has started in 12,2819 ms
[09:01:20 DBG] Removing outdated records from the 'AggregatedCounter' table...
[09:01:20 DBG] Execution loop SqlServerHeartbeatProcess:f57fa439 has started in 11,9128 ms
[09:01:20 DBG] Aggregating records in 'Counter' table...
[09:01:20 DBG] Execution loop Worker:e40656de has started in 12,6065 ms
[09:01:20 DBG] Execution loop Worker:aa196c8a has started in 18,7138 ms
[09:01:20 DBG] Execution loop Worker:f73ee2ec has started in 24,7742 ms
[09:01:20 DBG] Execution loop Worker:6c8a5c72 has started in 30,6857 ms
[09:01:20 INF] Server 55b4c516-d5a9-4516-be31-29b9a4e83f1c all the dispatchers started
[09:01:20 DBG] Execution loop Worker:21626abb has started in 37,1683 ms
[09:01:20 DBG] Execution loop DelayedJobScheduler:e419d23f has started in 12,6418 ms
[09:01:20 DBG] Execution loop RecurringJobScheduler:677d3276 has started in 12,5518 ms
[09:01:20 DBG] Removing outdated records from the 'Job' table...
[09:01:20 DBG] Removing outdated records from the 'List' table...
[09:01:20 DBG] Removing outdated records from the 'Set' table...
[09:01:20 DBG] Removing outdated records from the 'Hash' table...
[09:01:50 DBG] Server 55b4c516-d5a9-4516-be31-29b9a4e83f1c heartbeat successfully sent
[09:02:20 DBG] Server 55b4c516-d5a9-4516-be31-29b9a4e83f1c heartbeat successfully sent
[09:02:50 DBG] Server 55b4c516-d5a9-4516-be31-29b9a4e83f1c heartbeat successfully sent
[09:03:20 DBG] Server 55b4c516-d5a9-4516-be31-29b9a4e83f1c heartbeat successfully sent
[09:03:50 DBG] Server 55b4c516-d5a9-4516-be31-29b9a4e83f1c heartbeat successfully sent
[09:04:20 DBG] Server 55b4c516-d5a9-4516-be31-29b9a4e83f1c heartbeat successfully sent
[09:04:50 DBG] Server 55b4c516-d5a9-4516-be31-29b9a4e83f1c heartbeat successfully sent
[09:05:20 DBG] Server 55b4c516-d5a9-4516-be31-29b9a4e83f1c heartbeat successfully sent
[09:05:50 DBG] Server 55b4c516-d5a9-4516-be31-29b9a4e83f1c heartbeat successfully sent
[09:06:20 DBG] Aggregating records in 'Counter' table...
[09:06:20 DBG] Server 55b4c516-d5a9-4516-be31-29b9a4e83f1c heartbeat successfully sent
[09:06:50 DBG] Server 55b4c516-d5a9-4516-be31-29b9a4e83f1c heartbeat successfully sent
[09:07:20 DBG] Server 55b4c516-d5a9-4516-be31-29b9a4e83f1c heartbeat successfully sent
Here are the instances from the Hangfire.Server table:
Id Data LastHeartbeat
55b4c516-d5a9-4516-be31-29b9a4e83f1c
{"$type":"Hangfire.SqlServer.Entities.ServerData, Hangfire.SqlServer","WorkerCount":5,"Queues":{"$type":"System.String[], System.Private.CoreLib","$values":["default","default","notdefault","background","export","provisioning"]},"StartedAt":"2023-03-08T08:01:20.073434Z"}
2023-03-08 08:24:51.687
c7518906-d440-4113-85ad-0c1a97e01acc
{"$type":"Hangfire.SqlServer.Entities.ServerData, Hangfire.SqlServer","WorkerCount":5,"Queues":{"$type":"System.String[], System.Private.CoreLib","$values":["default","default","notdefault","background","export","provisioning"]},"StartedAt":"2023-03-08T08:01:32.2476057Z"}
2023-03-08 08:24:32.387
I've tested truncating all hangfire-related tables but also during a fresh run two instances are created.
I also tested writing an initializer where I deleted all, if any, servers before startup.
In this case the deployed application was stopped on the deployment server IIS, but the application pool was still running creating the additional server instance.