I'm currently dealing with a scenario where I need to adjust the number of replicas or partitions in Kafka while it's actively running. However, I noticed in Kafka UI that this operation is marked as a "danger zone."
Being relatively new to Kafka, my understanding is that altering the replica count impacts how messages are sent to Kafka during production, or it changes the partition number for message queuing and reception. I'm concerned about the potential risks involved in modifying the replica or partition numbers while Kafka is running.
Has anyone experienced issues, such as Kafka crashing, after making these changes during runtime?
I've conducted a simple test on my Kafka setup and didn't encounter any issues. However, I'm still concerned about the possibility of causing a crash in a production environment.
Are there any specific risks or concerns that I should be aware of before proceeding?