Migrating Azure Application Insights logging in a python project from OpenCensus to OpenTelemetary

Question

I am trying to move from OpenCensus to OpenTelemetry as the former project has been sunsetted and will no longer be supported by Microsoft later next year.

Current code to create log config:

log_config = {
    "version": 1,
    "disable_existing_loggers": True,
        "formatters": {
        "default": {
            "()": "uvicorn.logging.DefaultFormatter",
            "fmt": "%(levelprefix)s [%(thread)d] [%(asctime)s.%(msecs)03d] %(message)s",
            "datefmt": "%Y-%m-%d %H:%M:%S",

        },
    },
    "handlers": {
        "default": {
            "level": "INFO",
            "formatter": "default",
            "class": "logging.StreamHandler",
            "stream": "ext://sys.stderr",
        },
        "azure": {
            "level": "WARNING",
            "class": "opencensus.ext.azure.log_exporter.AzureLogHandler",
            "instrumentation_key": "<appinsights-connection-string>",
        }
    },
    "loggers": {
        "": {"handlers": ["default", "azure"], "level": "INFO"},
    },
}

Current code to apply log config:

import logging
from logging.config import dictConfig
from log_config import log_config


dictConfig(log_config)

The sunsetting article states that:

We are excited to announce that OpenTelemetry has reached feature parity with OpenCensus in C++, .NET, Go, Java, JavaScript, PHP and Python.

So I thought it would be a drop in solution but I can't seem to find and code sample that looks like what I am currently doing. Is there an OpenTelemetry handler that I can use to log to AppInsights or is there another approach I can use? The Microsoft page specifically advises against using the opentelemetry-opencensus-shim.

Answer 1

Use OpenTelemetry to export traces to Azure Application Insights using the AzureMonitorTraceExporter. To log data to Azure Application Insights using OpenTelemetry, you don’t need a specific “handler” like you would in traditional logging.

Reference Used:

• Azure-monitor-opentelemetry-exporter .

• Enable Azure Monitor OpenTelemetry for .NET, Java, Node.js, and Python applications.

import  logging

from  logging.config  import  dictConfig

from opentelemetry import  trace

from  opentelemetry.sdk.trace  import  TracerProvider

from  opentelemetry.sdk.trace.export  import  BatchSpanProcessor

from  azure.monitor.opentelemetry.exporter  import  AzureMonitorTraceExporter

  

# Initialize OpenTelemetry Tracer Provider

tracer_provider = TracerProvider()

  

# Replace 'YOUR_CONNECTION_STRING' with your actual Application Insights connection string

connection_string = " "

  

# Create an Azure Monitor Trace Exporter with the connection string

exporter = AzureMonitorTraceExporter(connection_string=connection_string)

  

# Create a BatchSpanProcessor and add the exporter

span_processor = BatchSpanProcessor(exporter)

tracer_provider.add_span_processor(span_processor)

  

# Set the tracer provider

trace.set_tracer_provider(tracer_provider)

  

# Your logging configuration remains the same

log_config = {

"version":  1,

"disable_existing_loggers":  True,

"formatters":  {

"default":  {

"()":  "uvicorn.logging.DefaultFormatter",

"fmt":  "%(levelprefix)s [%(thread)d] [%(asctime)s.%(msecs)03d] %(message)s",

"datefmt":  "%Y-%m-%d %H:%M:%S",

},

},

"handlers":  {

"default":  {

"level":  "INFO",

"formatter":  "default",

"class":  "logging.StreamHandler",

"stream":  "ext://sys.stderr",

},

},

"loggers":  {

"":  {"handlers":  ["default"],  "level":  "INFO"},

},

}

  

# Apply the log configuration

dictConfig(log_config)

  

# Sample log messages

logger = logging.getLogger(__name__)

logger.info("This is an info message.")

logger.warning("This is a warning message.")

Output:

Answer 2

If you would like more control over configuration of your logging exporting pipeline, you can use the piece-wise exporter approach via the azure-monitor-opentelemetry-exporters.

It's important to note that if you want to collect logging telemetry and send to the TRACES table in application insights, you have to use the AzureMonitorLogExporter (not the AzureMonitorTraceExporter, which is for distributed tracing and will populate the requests/depenencies table in application insights).

Also keep in mind that the logging exporter through this exporter package is in beta, so use it to your discretion. Our official recommendation from Azure Monitor is to use the azure monitor opentelemetry distro which is a one-stop-shop for your telemetry needs to send data to application insights, and the package is stable.

A similar tracking issue can be found here: https://github.com/Azure/azure-sdk-for-python/issues/32322. If you have further questions feel free to comment on the issue as the github issues are more actively triaged and monitored.