I have two identical servers, I have setup elastic search on one and imported about 200m records into it. I now want the other server to have the same set of records and the two to work as a pair, just incase one server goes down I can quickly remove the one from the cluster and it will just work, no new data is inserted its just read. I followed the information I have found online and setup my hosts to find each other. It just seems like the indexes are not transferring over to the other server I see in the status of the shards that they are replicating, but they never increase in size in the data directory.
I have used this as a pointer.
How to add a new node to my Elasticsearch cluster
My question is can I just scp copy over the 300gb plus of data to the data directory or does elasticsearch need to do that?
Debug Info
curl 192.168.1.1:9200/_cat/recovery?v
index shard time type stage source_host target_host repository snapshot files files_percent bytes bytes_percent total_files total_bytes translog translog_percent total_translog
pelias 0 248094 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 76 13092952848 0 100.0% 0
pelias 0 333911 relocation index 192.168.1.2 192.168.1.1 n/a n/a 76 76.3% 13092952848 9.6% 76 13092952848 0 100.0% 0
pelias 1 248053 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 70 12924273847 0 100.0% 0
pelias 1 333915 relocation index 192.168.1.2 192.168.1.1 n/a n/a 70 98.6% 12924273847 98.0% 70 12924273847 0 100.0% 0
pelias 2 246409 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 88 13122252645 0 100.0% 0
pelias 3 247434 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 89 13020268507 0 100.0% 0
pelias 4 247109 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 91 13173644369 0 100.0% 0
pelias 5 247574 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 85 13133234395 0 100.0% 0
pelias 6 245236 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 82 12882617780 0 100.0% 0
pelias 7 246530 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 86 13098807983 0 100.0% 0
pelias 8 245089 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 73 12963045661 0 100.0% 0
pelias 9 245149 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 94 13072471481 0 100.0% 0
pelias 10 248994 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 85 12950316547 0 100.0% 0
pelias 11 247667 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 73 13329893241 0 100.0% 0
pelias 12 248055 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 79 12930959145 0 100.0% 0
pelias 13 249255 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 106 11586141290 0 100.0% 0
pelias 14 247759 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 82 15107208138 0 100.0% 0
pelias 15 247888 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 79 13112198101 0 100.0% 0
pelias 16 247675 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 91 14672189956 0 100.0% 0
pelias 17 247205 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 73 13048452056 0 100.0% 0
pelias 18 246910 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 82 13095873096 0 100.0% 0
pelias 19 246325 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 70 15019530308 0 100.0% 0
pelias 20 247924 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 77 12898451788 0 100.0% 0
pelias 21 248663 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 76 13035468700 0 100.0% 0
pelias 22 249105 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 95 13045543380 0 100.0% 0
pelias 23 246119 store done 192.168.1.2 192.168.1.2 n/a n/a 0 100.0% 0 100.0% 79 13052787518 0 100.0% 0
curl 192.168.1.1:9200/_cat/shards?v
index shard prirep state docs store ip node
pelias 22 p STARTED 21505006 12.1gb 192.168.1.2 ronin-tile02
pelias 5 p STARTED 21503012 12.2gb 192.168.1.2 ronin-tile02
pelias 2 p STARTED 21499705 12.2gb 192.168.1.2 ronin-tile02
pelias 15 p STARTED 21504176 12.2gb 192.168.1.2 ronin-tile02
pelias 4 p STARTED 21512286 12.2gb 192.168.1.2 ronin-tile02
pelias 16 p STARTED 21508361 13.6gb 192.168.1.2 ronin-tile02
pelias 20 p STARTED 21499566 12gb 192.168.1.2 ronin-tile02
pelias 6 p STARTED 21504264 11.9gb 192.168.1.2 ronin-tile02
pelias 14 p STARTED 21511288 14gb 192.168.1.2 ronin-tile02
pelias 23 p STARTED 21513431 12.1gb 192.168.1.2 ronin-tile02
pelias 13 p STARTED 21505815 10.7gb 192.168.1.2 ronin-tile02
pelias 12 p STARTED 21506998 12gb 192.168.1.2 ronin-tile02
pelias 10 p STARTED 21511704 12gb 192.168.1.2 ronin-tile02
pelias 9 p STARTED 21504736 12.1gb 192.168.1.2 ronin-tile02
pelias 1 p RELOCATING 21506366 12gb 192.168.1.2 ronin-tile02 -> 192.168.1.1 PsmIinmpQ5myXE5oNdw4qA ronin-tile01
pelias 7 p STARTED 21508186 12.1gb 192.168.1.2 ronin-tile02
pelias 21 p STARTED 21511618 12.1gb 192.168.1.2 ronin-tile02
pelias 17 p STARTED 21511014 12.1gb 192.168.1.2 ronin-tile02
pelias 18 p STARTED 21512577 12.1gb 192.168.1.2 ronin-tile02
pelias 8 p STARTED 21511765 12gb 192.168.1.2 ronin-tile02
pelias 11 p STARTED 21504697 12.4gb 192.168.1.2 ronin-tile02
pelias 3 p STARTED 21509465 12.1gb 192.168.1.2 ronin-tile02
pelias 19 p STARTED 21503146 13.9gb 192.168.1.2 ronin-tile02
pelias 0 p RELOCATING 21512876 12.1gb 192.168.1.2 ronin-tile02 -> 192.168.1.1 PsmIinmpQ5myXE5oNdw4qA ronin-tile01
curl 192.168.1.1:9200/_cat/indices?v
health status index pri rep docs.count docs.deleted store.size pri.store.size
green open pelias 24 0 516182058 89 295.5gb 295.5gb
In order to duplicate data between two (or more) Elasticsearch nodes, you need to add replicas. For your index (
pelias) and your host (192.168.1.1:9200), you can add a single replica with the followingcurlcommand:By the way, thanks for using Pelias, I'm one of the core maintainers :)
I also maintain a set of bash scripts to accomplish common Elasticsearch tasks here. There is a script specifically for adding replicas that you or others might find useful.