I'm running a Temporal workflow. and I need to send APM tracing to ELK stack (Elastic)

Elastic provides a java agent elastic-apm-agent to send APM tracing logs to ELK's APM server.

As per my understanding elastic-apm-agent uses OpenTelemetry for tracing. But Temporal uses OpenTracing. So I used a bridge OpenTracingShim for trace/context propagation.

But when I run my workflow it gives following exception

    at io.temporal.internal.statemachines.WorkflowStateMachines.createEventProcessingException(WorkflowStateMachines.java:373)
    at io.temporal.internal.statemachines.WorkflowStateMachines.handleEventsBatch(WorkflowStateMachines.java:297)
    at io.temporal.internal.statemachines.WorkflowStateMachines.handleEvent(WorkflowStateMachines.java:260)
    at io.temporal.internal.replay.ReplayWorkflowRunTaskHandler.applyServerHistory(ReplayWorkflowRunTaskHandler.java:249)
    at io.temporal.internal.replay.ReplayWorkflowRunTaskHandler.handleWorkflowTaskImpl(ReplayWorkflowRunTaskHandler.java:231)
    at io.temporal.internal.replay.ReplayWorkflowRunTaskHandler.handleWorkflowTask(ReplayWorkflowRunTaskHandler.java:165)
    at io.temporal.internal.replay.ReplayWorkflowTaskHandler.handleWorkflowTaskWithQuery(ReplayWorkflowTaskHandler.java:133)
    at io.temporal.internal.replay.ReplayWorkflowTaskHandler.handleWorkflowTask(ReplayWorkflowTaskHandler.java:98)
    at io.temporal.internal.worker.WorkflowWorker$TaskHandlerImpl.handleTask(WorkflowWorker.java:413)
    at io.temporal.internal.worker.WorkflowWorker$TaskHandlerImpl.handle(WorkflowWorker.java:320)
    at io.temporal.internal.worker.WorkflowWorker$TaskHandlerImpl.handle(WorkflowWorker.java:261)
    at io.temporal.internal.worker.PollTaskExecutor.lambda$process$0(PollTaskExecutor.java:105)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.RuntimeException: WorkflowTask: failure executing SCHEDULED->WORKFLOW_TASK_STARTED, transition history is [CREATED->WORKFLOW_TASK_SCHEDULED]
    at io.temporal.internal.statemachines.StateMachine.executeTransition(StateMachine.java:163)
    at io.temporal.internal.statemachines.StateMachine.handleHistoryEvent(StateMachine.java:103)
    at io.temporal.internal.statemachines.EntityStateMachineBase.handleEvent(EntityStateMachineBase.java:84)
    at io.temporal.internal.statemachines.WorkflowStateMachines.handleSingleEvent(WorkflowStateMachines.java:419)
    at io.temporal.internal.statemachines.WorkflowStateMachines.handleEventsBatch(WorkflowStateMachines.java:295)
    ... 13 common frames omitted
Caused by: java.lang.ClassCastException: class io.opentelemetry.api.internal.AutoValue_ImmutableSpanContext cannot be cast to class co.elastic.apm.agent.opentelemetry.tracing.OTelSpanContext (io.opentelemetry.api.internal.AutoValue_ImmutableSpanContext is in unnamed module of loader org.springframework.boot.loader.LaunchedURLClassLoader @78a8978a; co.elastic.apm.agent.opentelemetry.tracing.OTelSpanContext is in unnamed module of loader co.elastic.apm.agent.bci.classloading.IndyPluginClassLoader @7bfedfb7)
    at co.elastic.apm.agent.opentelemetry.tracing.OTelSpanBuilder.startSpan(OTelSpanBuilder.java:198)
    at io.opentelemetry.opentracingshim.SpanBuilderShim.start(SpanBuilderShim.java:227)
    at io.temporal.opentracing.internal.OpenTracingWorkflowInboundCallsInterceptor.execute(OpenTracingWorkflowInboundCallsInterceptor.java:71)
    at io.temporal.internal.sync.POJOWorkflowImplementationFactory$POJOWorkflowImplementation.execute(POJOWorkflowImplementationFactory.java:314)
    at io.temporal.internal.sync.WorkflowExecutionHandler.runWorkflowMethod(WorkflowExecutionHandler.java:70)
    at io.temporal.internal.sync.SyncWorkflow.lambda$start$0(SyncWorkflow.java:135)
    at io.temporal.internal.sync.CancellationScopeImpl.run(CancellationScopeImpl.java:102)
    at io.temporal.internal.sync.WorkflowThreadImpl$RunnableWrapper.run(WorkflowThreadImpl.java:107)
    at io.temporal.worker.ActiveThreadReportingExecutor.lambda$submit$0(ActiveThreadReportingExecutor.java:53)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    ... 3 common frames omitted

I have tried different Tracing method as mentioned on Temporal Community Forum

0

There are 0 best solutions below