What is the best way to automate the elasticsearch snapshot and restore

Question

I need to automate the snapshot and restore from one cluster to a backup cluster , but when I try to restore the snapshot it complains about the indices already exist. Then I either need to delete those indices or close those to be freshly restored. Is there any --force kind of option to overwrite everything from live cluster to backup cluster ?

There is re-indxeing option but that is slow as compared to snapshot and restore.

Answer 1

You can define rename_pattern and rename_replacement as documentation suggests. To make it fully automated you could add time/date:

POST /_snapshot/my_backup/snapshot_1/_restore
{
  "indices": "index_1,index_2",
  "ignore_unavailable": true,
  "include_global_state": true,
  "rename_pattern": "(.+)",
  "rename_replacement": "$1_20180820"
}

And then use aliases to make this "backup" index look like a "normal" one:

POST /_aliases
{
    "actions" : [
        { "add" : { "index" : "index_1_20180820", "alias" : "index_1" } }
    ]
}

Of course this means that you would have to write some automation scripts that generate that time/date and check the snapshot restore progress.

Hope that helps!