I am using ELK stack with Netflow module. First of all, when I checked CPU usage Logstash was using a lot of resources and I decided to stop it. This moment Elasticsearch/Kibana/Logstash is stopped. I mean, I ran command sudo service elasticsearch/kibana/logstash stop. Basically, I think that something is wrong with logstash. When I am see log in htop I am getting something like this, I do not understand why.
When checking logstash service status, getting something like this.
Logstash is still running, and I am trying to figure out how to stop it. I think, I ran it in a wrong manner at the start, but why not possible to stop it forever?
You have to be aware that Logstash will not stop unless it was able to end all pipelines and got rid of all the events in them.
Stopping usually means that it will stop the input, making it so that no new events will enter the pipelines, then depending on the config of persistent queues or not it will process what is in the queue or not. This can indeed take upto several minutes depending on the amount of events and how hard the processing exactly is.
Also keep in mind that when you have large bulk requests going to Elasticsearch itself it could mean that the messages are getting too large.
If it is really needed to stop the Logstash and there is really no need to keeping the events that are in the queue, you can always do a
kill -9on the pid.