I am starting with kafka , have a question on the consumer groups. We have an application where we want different consumers from same group subscribing to different topics. The grouping is done based on some business criteria . To be specific consumer 1 from group A and consumer 2 from group A are subscribed to Topic 1 and Topic 2 each with 10 partitions. Does this mean that there consumer 1 can scale to 10 and consumer 2 also can scale to 10 since they are subscribed to different topics. Is this a correct design
Kafka Consumers subscribed to different topics in same consumer group
1k Views Asked by user2254608 At
1
There are 1 best solutions below
Related Questions in APACHE-KAFKA
- Spark streaming + kafka throughput
- How to diagnose Kafka topics failing globally to be found
- kafka: what do 'soTimeout', 'bufferSize' and 'minBytes' mean for SimpleConsumer?
- Fail to create SparkContext
- Syntax error on tokens, delete these tokens - kafka spring integration demo application
- How could Kafka 0.8.2.1 with offsets.storage=kafka still require ZooKeeper?
- Message Queues: Per Message Guarantees
- How should a Kafka HLC figure out the # of partitions for a topic?
- Kafka multiple consumers for a partition
- Should Apache Kafka and Hadoop be installed seperatedly (on a diffrent cluster)?
- how does one combine kafka-node producer and node tail?
- How to fix NoClassDefFoundError with custom Kafka producer under Eclipse?
- Apache Samza's CheckpointTool won't give away partition offsets
- Offsets for Kafka Direct Approach in Spark 1.3.1
- Simulate kafka broker failures in multi node kafka cluster and what operations and tools to use to mitigate data loss issues
Related Questions in CONSUMER
- How should I use Consumer object in oauth2 v.2 in Python 3.4?
- Declare Consumer and action with it in one line
- Why does this consumer (running on a servlet) not receive any messages from the kafka topic?
- Topic can't be found when producing messages: UNKNOWN_TOPIC_OR_PARTITION
- Trying to solve consumer-producer in java with multithreading
- One JMS Consumer stops listening active mq topic while second do not
- Hazelcast Queue ItemListener for Pending Item (Producer-Consumer Concept)
- Trying to pass variables to child process
- How can clang avoid to analyze the builtin type/function in C/C++?
- How to use execl() to execute another program
- Does OSGI bundle service consumer need to create "stub" of service interface?
- HornetQ: Configure the number of consumers
- RabbitMq Consumers do Not Consume
- Kafka Consumer poll messages with python
- RabbitMQ - Consume multiple queues by priority
Related Questions in KAFKA-TOPIC
- How Kafka guarantee the messages order while we increase the partitions in runtime?
- Apache Flink - Partitioning the stream equally as the input Kafka topic
- Does Kafka chose an alternative partition in the same topic if the partition I want to send message to was full of disk?
- What happens to the offsets that are compacted in a kafka partition?
- Does Kafka consumer reads the message from active segment in the partition?
- after kafkaTemplate.send(topic, avdlObject); appends dataType in value
- In Kafka, the topic is not getting leader after recreation post deletion
- How to send headers from Pentaho DI to Kafka?
- How to access Kafka through nodeport
- Can compacted Kafka topic be used as key-value database?
- Kafka topic returns 0 messages
- Where is offset of consumer stored in Kafka
- Kafka: auto create topic with options
- Decrease topic replication factor after Kafka brokers removed from cluster and failed reassignments
- How to programmatically create a topic in Apache Kafka using C++ client librdkafka
Trending Questions
- UIImageView Frame Doesn't Reflect Constraints
- Is it possible to use adb commands to click on a view by finding its ID?
- How to create a new web character symbol recognizable by html/javascript?
- Why isn't my CSS3 animation smooth in Google Chrome (but very smooth on other browsers)?
- Heap Gives Page Fault
- Connect ffmpeg to Visual Studio 2008
- Both Object- and ValueAnimator jumps when Duration is set above API LvL 24
- How to avoid default initialization of objects in std::vector?
- second argument of the command line arguments in a format other than char** argv or char* argv[]
- How to improve efficiency of algorithm which generates next lexicographic permutation?
- Navigating to the another actvity app getting crash in android
- How to read the particular message format in android and store in sqlite database?
- Resetting inventory status after order is cancelled
- Efficiently compute powers of X in SSE/AVX
- Insert into an external database using ajax and php : POST 500 (Internal Server Error)
Popular Questions
- How do I undo the most recent local commits in Git?
- How can I remove a specific item from an array in JavaScript?
- How do I delete a Git branch locally and remotely?
- Find all files containing a specific text (string) on Linux?
- How do I revert a Git repository to a previous commit?
- How do I create an HTML button that acts like a link?
- How do I check out a remote Git branch?
- How do I force "git pull" to overwrite local files?
- How do I list all files of a directory?
- How to check whether a string contains a substring in JavaScript?
- How do I redirect to another webpage?
- How can I iterate over rows in a Pandas DataFrame?
- How do I convert a String to an int in Java?
- Does Python have a string 'contains' substring method?
- How do I check if a string contains a specific word?
yes, since within a topic kafka try to assign partitions to consumers as equal as possible. The key is
topic:consumer_group_idso it doesn't matteranother_topic:same_consumer_group_id- it's another key, and consumer withtopic:consumer_group_idcould be scaled to partitions number