How to add EventSource to a web application

Question

We finally got EventSource and ElasticSearch correctly configured in our service fabric cluster. Now that we have that we want to add EventSources to our web applications that interact with our service fabric applications so that we can view all events (application logs) in one location and filter / query via Kibana.

Our issue seems to be related to the differences between a service fabric app which is an exe and a .NET 4.6 (not .net CORE) web app which is stateless. In service Fabric we place the using statement that instantiates the pipeline in Program.cs and set an infinite sleep.

private static void Main()
{
        try
        {
            using (var diagnosticsPipeline = ServiceFabricDiagnosticPipelineFactory.CreatePipeline("CacheApp-CacheAPI-DiagnosticsPipeline"))
            {
                ServiceEventSource.Current.ServiceTypeRegistered(Process.GetCurrentProcess().Id, typeof(Endpoint).Name);
                // Prevents this host process from terminating so services keeps running. 
                Thread.Sleep(Timeout.Infinite);
            }

How do I do this in a web app? This is the pipeline code we are using for a non ServiceFabric implementation of the EventSource. This is what we are using:

using (var pipeline = DiagnosticPipelineFactory.CreatePipeline("eventFlowConfig.json"))
    {
        IEnumerable ie = System.Diagnostics.Tracing.EventSource.GetSources();

        ServiceEventSource.Current.Message("initialize eventsource");

    } 

We are able to see the pipeline and send events to ElasticSearch from within the using statement but not outside of it. So the question is:

1. how/where do we place our pipeline using statement for a web app?
2. Do we need to instantiate and destroy the pipeline that every time we log or is there a way to reuse that pipeline across the stateless web events? It would seem like that would be very expensive and hurt performance. Maybe we can we cache a pipeline?

That’s the jist, let me know if you need clarification. I see lots of doco out there for client apps but not much for web apps.

Thanks, Greg

UPDATE WITH SOLUTION CODE

DiagnosticPipeline pipeline;

protected void Application_Start(Object sender, EventArgs e)
{
    try
    {
        pipeline = DiagnosticPipelineFactory.CreatePipeline("eventFlowConfig.json");   
        IEnumerable ie = System.Diagnostics.Tracing.EventSource.GetSources();
        AppEventSource.Current.Message("initialize eventsource");

    }
}

protected void Application_End(Object sender, EventArgs e)
{
    pipeline.Dispose();
}

Answer 1

Assuming ASP.NET Core the simplest way to initialize EventFlow pipeline would be in the Program.cs Main() method, for example:

public static void Main(string[] args)
{
    using (var pipeline = DiagnosticPipelineFactory.CreatePipeline("eventFlowConfig.json"))
    {
        var host = new WebHostBuilder()
            .UseKestrel()
            .UseContentRoot(Directory.GetCurrentDirectory())
            .UseIISIntegration()
            .UseStartup<Startup>()
            .UseApplicationInsights()
            .Build();

            host.Run();
    }
}

This takes advantage of the fact that host.Run() will block until the server is shut down, and so the pipeline will exist during the time when requests are received and served.

Depending on the web framework you use things might vary. E.g. if the one you use offers "setup" and "cleanup" hooks, you could create a diagnostic pipeline during setup phase (and store a reference to it in some member variable), then dispose of it during cleanup phase. For example, in ASP.NET classic you'd put the code in global.asax.cs and leverage Application_OnStart and Application_OnEnd methods. See Application Instances, Application Events, and Application State in ASP.NET for details.

Creating a pipeline instance every time a request is served is quite inefficient, like you said. There is really no good reason to do that.