Elasticsearch Marvel set .marvel-2015-* indices to max number

Question

Hey I am using marvel alongside elasticsearch and I am trying to avoid using curator to clean indices that look like ".marvel-2015-*" is there a specific config or set of configs that I can use to accomplish this.

Note: I am using chef to provision the node and inside of the logstash cookbook I am setting the attribute in default.rb like so

default['logstash']['instance_default']['curator_days_to_keep'] = 14

I would assume this sets the max amount of these indices to 14. But when I added some fake ".marvel-2015-*" indices they still appear and are not cleared out.

I realize that I am talking about a tool for working with marvel curator and marvel itself, but I am new to these tools and I need help connecting these dots.

Ideally I want marvel to have the logic to just remove these indices by itself, and I don't know if there is some option to accomplish this in the plugins/marvel/marvel-1.3.1.jar

Any help would be appreciated.

Answer 1

I agree that ideally Marvel should provide this as a configuration option but, at time of writing, it does not and over time, the marvel indexes can become quite big. Here's an example for a cluster I'm currently managing:

I know you want to avoid using Curator but short of writing your own script or plugin to manage this, it is by far the easiest way to deal with this.

To purge Marvel indexes older than 30 days, you can do:

curator delete indices --timestring '%Y.%m.%d' --prefix '.marvel-2' --older-than 30 --time-unit 'days'

To test what would be deleted, I recommend to first use --dry-run :

curator --dry-run delete indices --timestring '%Y.%m.%d' --prefix '.marvel-2' --older-than 30 --time-unit 'days'