Reading data from _transaction_state topic in Kafka 0.11.0.1

Question

I want to read meta data for transaction (which is supported in Kafka 0.11.0.1) so that I can figure out that whether the transaction for a particular transactional ID has been committed or not. Currently I am getting the key and value from _transactional_state topic but it is in some encoded format. Below are some same key/value I received when I polled __transaction_state topic: key = 10000000mmm, value = � �����+'���������)

Answer 1

You can look to source code of TransactionLogMessageParser class inside kafka/tools/DumpLogSegments.scala file as an example. It uses readTxnRecordValue function from TransactionLog class. The first argument for this function could be retrieved via readTxnRecordKey function of the same class.

Answer 2

just like what has been done in Kafka how to read from __consumer_offsets topic

echo "exclude.internal.topics=false" > consumer.config

kafka-console-consumer --consumer.config consumer.config --formatter "kafka.coordinator.transaction.TransactionLog\$TransactionLogMessageFormatter" --bootstrap-server localhost:9092 --topic __transaction_state --from-beginning