ElasticSearch Rails resource_already_exists_exception when running tests

Question

I'm trying to run one of my tests, which makes a search, trying to assert the inclusion of records in the search result, but in the meantime, I'm receiving a Elasticsearch::Transport::Transport::Errors::BadRequest error:

SearchTest#test_simple_test_returns_product:
Elasticsearch::Transport::Transport::Errors::BadRequest: [400] 

{
  "error":{
    "root_cause":[
      {
        "type":"resource_already_exists_exception",
        "reason":"index [app_application_test_products/FTt1YC6eQrCw2XwJuqjmDw] already exists",
        "index_uuid":"FTt1YC6eQrCw2XwJuqjmDw",
        "index":"app_application_test_products"
      }
    ],
    "type":"resource_already_exists_exception",
    "reason":"index [app_application_test_products/FTt1YC6eQrCw2XwJuqjmDw] already exists",
    "index_uuid":"FTt1YC6eQrCw2XwJuqjmDw",
    "index":"app_application_test_products"
  },
  "status":400
}

When I perform a search in development, it works as expected, but in tests is throwing such error, within the test I've added an import and an index refresh, nothing else:

class SearchTest < ActiveSupport::TestCase
  setup do
    Product.import force: true
    Product.__elasticsearch__.refresh_index!
  end

  test "simple test returns product" do
    product = products(:one)
    I18n.locale = product.market.lang
    search = Search.new(
      category: product.category.custom_slug,
      page: 1,
      market_id: product.market_id,
      status: "active",
      seed: Date.today.to_time.to_i
    )
    assert_includes search.results.records, products(:one)
    assert_includes search.results.records, products(:two)
    assert_not_includes search.results.records, products(:three)
  end
end

Any help is appreciated, as any hint to improve the code.

I'm using:

# Gemfile
gem 'minitest', '5.10.1'

# Gemfile.lock
elasticsearch (6.1.0)
elasticsearch-model (6.0.0)
elasticsearch-rails (6.0.0)
minitest (= 5.10.1)

Answer 1

I had the wrong mappings in my model. Instead using the type option, I was using index, what made ElasticSearch to create a multiple mapping. Which isn't available since the version 6.4 (I guess).

Answer 2

I was using time freezing in multiple specs, so creating a new object with the same created_at time as the object in the previous spec was causing the resource_already_exists_exception error. Slightly adjusting the timestamp to freeze for each spec fixed the problem.

Answer 3

I'm glad you found the root cause for your specific issue.

I ran into a similar issue with ruby-on-rails gem for elasticsearch. While the mappings are all fine, i did get the exact same error message. Leaving my answer here so that anyone else who comes here can get more help.

After a lot of trying and error, eventually figured out that the reason is that it's timing out on the create index.

If you change the client timeout to 60 seconds (it failed with 30 seconds), it was able to create the index successfully without causing this intermittent error.

connection_hash = {
      hosts: [ "localhost:9220" ]
      reload_connections: true
      adapter: :httpclient
      retry_on_failure: 2
      request_timeout: 60
}

es_connection_client = Elasticsearch::Client.new(connection_hash)

Also, found this issue that is related and was closed after a similar answer. https://github.com/ankane/searchkick/issues/843#issuecomment-384136164