Elastic Search bulk indexing using perl

Question

I have tried the bulk API Perl client for content indexing in Elasticsearch. I am getting Error on the Bulk Ingestion line. Please find the code below:

my $ifileid=0;
my $dir = '/home/bala/input_files/output';
opendir(DIR, $dir) or die $!;
my @arfiles = readdir (DIR);
closedir(DIR);
print scalar @arfiles." Total files\n";
foreach(@arfiles)
{
    my $file = $_;
    if ($ifileid>1)
    {
    $doc = {index => 'my_index', type => 'blog_post', id => $ifileid, body => {filename => $file, content => 'bala'}}; 
    push @docs, { create => $doc };
    if ($ibulkid==100)
        {       
            # bulk index docs
           my $res = $e->bulk(\@docs);      
           if ( $res->{errors} ) 
           {
            die "Bulk index had issues: " . $json->encode( $res->{errors} );
           }
           $ibulkid=0;      
        }
        $ibulkid++;
    }
    $ifileid++;
}

I am getting the following error:

Error => Not a HASH reference at /usr/local/share/perl5/Search/Elasticsearch/Role/Client/Direct.pm line 15.

Answer 1

The above usage of bulk api is wrong. bulk takes as input a hashref where the body is a reference to array of actions and documents

For example something on these lines should work:

$action = {index => {_index => 'my_index', _type => 'blog_post', _id => $ifileid}};
$doc =  {filename => $file, content => 'bala'};
push @docs, $action;
push @docs,$doc
if ($ibulkid==100)
    {       
        # bulk index docs
        my $res = $e->bulk(body => \@docs);      
       if ( $res->{errors} ) 
       {
        die "Bulk index had issues: " . $json->encode( $res->{errors} );
       }
       $ibulkid=0;      
    }
    $ibulkid++;
}
$ifileid++;