Spring Boot 3 KafkaTemplate tracing (traceId, spanId, baggage) is missing

Question

Prior to Spring Boot 3, tracing, including Kafka tracing, was performed using Sleuth. It was possible to propagate a tracing context (traceId, spanId, baggage) over Kafka.

Sleuth no longer exists. Spring Boot now takes care of these aspects.

It seems that tracing on the producer side (KafkaTemplate) disappeared.

This was previously handled using a template post-processor. See KafkaFactoryBeanPostProcessor, TraceProducerPostProcessor and TracingConsumerPostProcessor for more technical details

https://github.com/spring-cloud/spring-cloud-sleuth/tree/80f0889c91d337e84ce0eca2d602464a1a4355c6/spring-cloud-sleuth-brave/src/main/java/org/springframework/cloud/sleuth/brave/instrument/messaging

Have I forgotten something? Should I configure it manually or import another dependency?

Any comments or feedback are more than welcome on this tricky topic.

EDIT 1:

By adding brave-instrumentation-kafka-clients into the classpath, it's also possible to setup it manually using the following portion of code

@Bean
public <K, V> ProducerFactory<K, V> kafkaProducerFactory(KafkaProperties kafkaProperties, ObjectProvider<Tracing> tracing) {
    final var factory = new DefaultKafkaProducerFactory<>(kafkaProperties.buildProducerProperties());
        tracing.ifAvailable(t -> {
            final var kafkaTracing = KafkaTracing.create(t);
            factory.addPostProcessor(kafkaTracing::producer);
        });
        return factory;
}

Answer 1

The observation is not enabled on Spring Kafka components by default.

See KafkaTemplate.setObservationEnabled(true).

More in docs: https://docs.spring.io/spring-kafka/reference/3.1-SNAPSHOT/kafka/micrometer.html#observation

Answer 2

Finally found the issue. The observation support is working fine when enabled.

I used the setter on the template to enable it but the property mapper used in the auto configuration was resetting the value to false.

    public KafkaTemplate<?, ?> kafkaTemplate(KafkaProperties kafkaProperties,
                                         ObservationRegistry observationRegistry,
                                         ProducerFactory<Object, Object> kafkaProducerFactory,
                                         ProducerListener<Object, Object> kafkaProducerListener,
                                         ObjectProvider<RecordMessageConverter> messageConverter) {
    PropertyMapper map = PropertyMapper.get().alwaysApplyingWhenNonNull();
    KafkaTemplate<Object, Object> kafkaTemplate = new KafkaTemplate<>(kafkaProducerFactory);

    kafkaTemplate.setObservationEnabled(true);
    messageConverter.ifUnique(kafkaTemplate::setMessageConverter);
    map.from(kafkaProducerListener).to(kafkaTemplate::setProducerListener);
    map.from(kafkaProperties.getTemplate().getDefaultTopic()).to(kafkaTemplate::setDefaultTopic);
    map.from(kafkaProperties.getTemplate().getTransactionIdPrefix()).to(kafkaTemplate::setTransactionIdPrefix);
    map.from(kafkaProperties.getTemplate().isObservationEnabled()).to(kafkaTemplate::setObservationEnabled); // RESET THE FLAG TO FALSE IF NOT SET INTO THE PROPERTIES
    return kafkaTemplate;
}

So, in brief, this was due to a wrong re-definition of the KafkaTemplate ("copy pasted" from the default auto configuration) into a custom auto-configuration. I didn't pay attention to the PropertyMapper.