I m trying to implement a simple POC for a real time streaming application based on event driven architecture using an event streaming platform ( Kafka, Apache Pulsar, Rabbit stream) not fixed yet.
I m confused at the point that my application can have many requets with the same type but different content (let's say sent by a Web Applcation), that should return foreach request a stream with responses updates.
Example :
Let's say that I have two services A and B, Service A send an event to the message broker to retrieve a Spot for a given Currency Pair (EURUSD), then Service B will receive this event and send a SpotPriced event to the Broker, finally Service A will need to handle this event and apply some logic to stream a final price and send updates when spot get updated.
Service A --> (Spot) --->Broker---> Service B---->(SpotPriced)---->Borker---->Service A-->Apply logic ----> Send stream of updates
Well I have two approches :
1-Use a fire and forget approach with a topic for any new request to handle it's stream separatly, but this will be costly in terms of perfomances due the numbers of topics that will be created, and also complicated in term of topics naming espacially that requests can have many details to identify them.
2- Try to use an Asynchronous Request-Reply pattern with dedicated 2 topics for requests and responses, accompagned with a correlation-id to match the requets and their responses in the services side, I know this is contradictory with the goal of asynchronous data streaming ( Kafka, Pulsar, rabbitMQ, ...), and it will need extra effort to handle services failure.
Advices will be appreciated :D