Importing into meilisearch from mongodb using laravel scout takes too long

832 Views Asked by At

I have around 6 million rows in my mongodb collection and importing into meilisearch using php artisan scout:import 'model' takes forever to finish.
Importing data with limit option php artisan scout:import 'model' -c 10000 gives me the following error.

MongoDB\Exception\InvalidArgumentException Expected "limit" option to have type "integer" but found "string" at vendor/mongodb/mongodb/src/Exception/InvalidArgumentException.php:59 55▕ 56▕ $expectedType = $typeString; 57▕ } 58▕ ➜ 59▕ return new static(sprintf('Expected %s to have type "%s" but found >"%s"', $name, $expectedType, get_debug_type($value))); 60▕ } 61▕ } 62▕

+27 vendor frames 28 artisan:37 Illuminate\Foundation\Console\Kernel::handle()

I also tried exporting the collection as json from mongodb and manual importing into meilisearch using curl -X POST 'http://127.0.0.1:7700/indexs/posts/documents' / --data @/data/posts.json gives the following error.

{"message":"Invalid JSON: invalid type: map, expected a sequence at line 1 column 0","errorCode":"bad_request","errorType":"invalid_request_error","errorLink":"https://docs.meilisearch.com/errors#bad_request"}curl: (3) URL using bad/illegal format or missing URL Posts.json is the exported json file of mongodb collection using mongoexport command.

How can I import data fast into meilisearch?

Versions

  • "laravel/scout":"^9.1"
  • "laravel/framework": "^8.12",
  • "meilisearch/meilisearch-php": "^0.18.2",
  • mongodb version : "3.6"

OS

  • Ubuntu 20.04
0

There are 0 best solutions below