CakePHP 4, AWS Elasticsearch 7 - The security token included in the request is invalid

Question

So I'm running CakePHP 4 on an EC2 instance, AWS ES 7 and I've setup the ElasticSearch plugin in CakePHP.

composer require cakephp/elastic-search "^3.0"

I've added the elastic datasource connection in config/app.php

'elastic' => [
            'className' => 'Cake\ElasticSearch\Datasource\Connection',
            'driver' => 'Cake\ElasticSearch\Datasource\Connection',
            'host' => 'search-DOMAIN.REGION.es.amazonaws.com',
            'port' => 443,
            'transport' => "AwsAuthV4",
            'aws_access_key_id' => "KEY",
            'aws_secret_access_key' => "SECRET",
            'aws_region' => "REGION",
            'ssl' => 'true',
        ],

.. and I've activated the plugin

use Cake\ElasticSearch\Plugin as ElasticSearchPlugin;

class Application extends BaseApplication
{
    public function bootstrap()
    {
        $this->addPlugin(ElasticSearchPlugin::class);

I've manually added 1 index record to ES via curl from the EC2 instance. So I know the communication between EC2 and ES works.

curl -XPUT -u 'KEY:SECRET' 'https://search-DOMAIN.REGION.es.amazonaws.com/movies?pretty' -d '{"director": "Burton, Tim", "genre": ["Comedy","Sci-Fi"], "year": 1996, "actor": ["Jack Nicholson","Pierce Brosnan","Sarah Jessica Parker"], "title": "Mars Attacks!"}' -H 'Content-Type: application/json'

I also managed to search for this record via curl without any problems.

In the AppController.php I tried this simple search just to see if the plugin works and for the life of me, I can't get it to work.

# /src/Controller/AppController.php
...
use Cake\ElasticSearch\IndexRegistry;

class AppController extends Controller
{

    public function initialize(): void
    {
        parent::initialize();

        $this->loadModel('movies', 'Elastic');
        $query = $this->movies->find('all');
        $results = $query->toArray();

I'm getting the following error:

Client error: POST https://search-DOMAIN.REGION.es.amazonaws.com/movies/movies/_search resulted in a 403 Forbidden response: {"message":"The security token included in the request is invalid."} Elastica\Exception\Connection\GuzzleException

Seems like the plugin adds the 'Index' name twice for some reason. I looked everywhere for a setting that I might have missed. If I copy and paste the above URL and remove the duplicate Index from the URL in a browser it works fine.

https://search-DOMAIN.REGION.es.amazonaws.com/movies/_search

Am I missing something here?

I've even tried this method, and I get the same problem with duplicated index values in the url.

$Movies = IndexRegistry::get('movies');
$query = $Movies->find('all');
$results = $query->toArray();

I've tried a new/clean CakePHP instance and I get the same problem? Is there something wrong with the plugin? Is there a better approach to communicate with ES via CakePHP?

Answer 1

I'm not familiar with the plugin and Elasticsearch, but as far as I understand, one of the movies is the index name, and one of them is the type name, where the type name - at least according to the documentation - should be singular, ie the path would instead be expected to look like:

/movies/movie/_search

Furthermore, Index classes assume that the type mapping has the singular name of the index. For example the articles index has a type mapping of article.

https://book.cakephp.org/elasticsearch/3/en/3-0-upgrade-guide.html#types-renamed-to-indexes

Whether that would be the correct path with respect to what is supported by the used Elasticsearch version, that might be a different question.

You may want to open an issue over at GitHub.