Windows Service - try running until run successfully

Question

I have an Windows Service, that connects to database. If database is not accesable it throws exception and stops. I want it to try and run this service in loop, until it is started succesfully (maybe for some period of time, but it doesn't matter now). I have a code like this:

public class Program
{
    public static void Main(string[] args)
    {
        var appConfigurationManager = new AppConfigurationManager();
        bool startupSuccessful = false;

        do
        {
            using (var host = CreateHostBuilder(args, appConfigurationManager).Build())
            try
            {

                host.Run();
                startupSuccessful = true;
            }
            catch (Exception ex)
            {
                Thread.Sleep(TimeSpan.FromSeconds(30));
            }
        }
        while (!startupSuccessful);
    }

    public static IHostBuilder CreateHostBuilder(string[] args, IAppConfigurationManager appConfigurationManager)
    {
        return Host.CreateDefaultBuilder(args)
            .UseWindowsService(options =>
            {
                options.ServiceName = $"MyServiceName";
            })
            .ConfigureServices((context, services) =>
            {
                .
                .
                .

                services.AddHostedService<SomeBackgroundService>();

                .
                .
                .
            });
    }
}

I install the service, disable my database and started it manually. I got error that database is not accessible, then program waited for 30s and after that, when it tries to run the Host second time there's an error:

Service cannot be started. An instance of the service is already running..

After that, when I am stopping service manually it crushes with an error:

Failed to stop service. System.AggregateException: An error occurred while writing to logger(s). (Cannot access a disposed object. Object name: 'EventLogInternal'.) ---> System.ObjectDisposedException: Cannot access a disposed object. Object name: 'EventLogInternal'.
at System.Diagnostics.EventLogInternal.OpenForWrite(String currentMachineName) at System.Diagnostics.EventLogInternal.InternalWriteEvent(UInt32 eventID, UInt16 category, EventLogEntryType type, String[] strings, Byte[] rawData, String currentMachineName) at System.Diagnostics.EventLogInternal.WriteEvent(EventInstance instance, Byte[] data, Object[] values) at System.Diagnostics.EventLog.WriteEvent(EventInstance instance, Byte[] data, Object[] values) at System.Diagnostics.EventLog.WriteEvent(EventInstance instance, Object[] values) at Microsoft.Extensions.Logging.EventLog.WindowsEventLog.WriteEntry(String message, EventLogEntryType type, Int32 eventID, Int16 category) at Microsoft.Extensions.Logging.EventLog.EventLogLogger.WriteMessage(String message, EventLogEntryType eventLogEntryType, Int32 eventId) at Microsoft.Extensions.Logging.EventLog.EventLogLogger.Log[TState](LogLevel logLevel, EventId eventId, TState state, Exception exception, Func3 formatter) at Microsoft.Extensions.Logging.Logger.<Log>g__LoggerLog|13_0[TState](LogLevel logLevel, EventId eventId, ILogger logger, Exception exception, Func3 formatter, List1& exceptions, TState& state) --- End of inner exception stack trace --- at Microsoft.Extensions.Logging.Logger.ThrowLoggingError(List1 exceptions) at Microsoft.Extensions.Logging.Logger.Log[TState](LogLevel logLevel, EventId eventId, TState state, Exception exception, Func3 formatter) at Microsoft.Extensions.Logging.Logger1.Microsoft.Extensions.Logging.ILogger.Log[TState](LogLevel logLevel, EventId eventId, TState state, Exception exception, Func`3 formatter) at Microsoft.Extensions.Logging.LoggerExtensions.Log(ILogger logger, LogLevel logLevel, EventId eventId, Exception exception, String message, Object[] args) at Microsoft.Extensions.Hosting.Internal.HostingLoggerExtensions.ApplicationError(ILogger logger, EventId eventId, String message, Exception exception) at Microsoft.Extensions.Hosting.Internal.ApplicationLifetime.StopApplication() at Microsoft.Extensions.Hosting.WindowsServices.WindowsServiceLifetime.OnStop() at System.ServiceProcess.ServiceBase.DeferredStop()

It looks like something is not well disposed, but I have no idea what else can I do with that. Maybe I got it all wrong and there's some other way to ensure that service won't stop when some error occurs and will try to run again?

Answer 1

@garyore The design of the service is important. Windows Services are intended to serve some APIs, or consist some Queues, or do some job on a schedule basis. It is not reasonable to have a service that connects to a database at the very first stage of running. Instead, your service can have a scheduled job, in the job, you do a recurring task based on the business logic, and during the task, it may connect to the database as some points. At that time, if the connection to the database is failed, you can issue an event on a monitoring system (e.g. ELK along with Grafana or any other monitoring system), and check for the issues. Why we write a service that requires to be connected to the database at the very first stage of start up?! So, with these in mind, I recommend you to do the following:

1. Create a scheduled job using any scheduling library (Quartz.Net, HangFire or else).
2. Add required Trace logs
3. Handle any exceptions, from business exceptions to technicals and log failures
4. Send a notification on incidents to the monitoring system when critical exceptions occure, so that a person is notified and takes any operations required (ITIL),
5. Try handling failures and create relisiency, using libraries like Polly.
6. The database should be available on a SLA basis, so that the service works as expected by the business. Try to find out why the database may fail to be available, and design Fault-Tolerance strategies for your workload.

If that is a project that you may need some collegues, I am open to cooperation :)

I invite you to have a look into my profile: https://www.linkedin.com/in/massoudsafari/

Best regards Massoud Safari