Building and querying state in Apache Kafka: Kafka Stream?

Question

I am building an apache cluster for my needs and most of it is stateless. But there is one situation because of which I really need state.

To explain lets say I am storing every Pharmacy store that opens and the transactions that happen at that and every store. So the store opens with an initial state of number of medicines. As medicines are sold and more medicines are stocked, the state continually keeps changing.

While Kafka is serving my need of keeping up with live transactions in real time I need to be able to build pharmacy store state and query and find out at any given point the count of a given medicine in a store. Is it possible? Is that what Kafka Stream is used for?

Answer 1

Yes, you can use Kafka Streams to build an application that consumes a Kafka topic and maintains a queryable store that is continuously updated to maintain, as in your example, the current drug inventory.

Check out the documentation to get started: http://docs.confluent.io/current/streams/index.html

Also check out these examples using Kafka Streams' "Interactive Queries" feature:

• https://github.com/confluentinc/examples/blob/3.1.x/kafka-streams/src/main/java/io/confluent/examples/streams/interactivequeries/WordCountInteractiveQueriesExample.java
• https://github.com/confluentinc/examples/blob/3.1.x/kafka-streams/src/main/java/io/confluent/examples/streams/interactivequeries/kafkamusic/KafkaMusicExample.java