NAME

Search::ElasticDirect - An interface to ElasticSearch API, version independent

VERSION

version 2.5.2

SYNOPSIS

use Search::ElasticDirect;

my $elk = Search::ElasticDirect->new( host=> 'SomeServer',  protocol=> 'http', authentication=> 'no' );
die "Connection error because $elk->{error}\n" if $elk->{error};

my $reply = $elk->direct( method=>'GET', path=>'_cat/nodes', param=>'h=name,ip,role,version&s=name:asc', format=>'data' );
die "$elk->{error}\n" if $elk->{error};

die "Allowed to run only at active master\n" unless $elk->ActiveMaster;
die $elk->{error} if $elk->{error};

foreach my $srv (@{$reply}) {
say join ',', $srv->{name}, $srv->{ip}, $srv->{version}, $srv->{role}
}

DESCRIPTION

An interface to ElasticSearch API, version independent

The DeepPagination methods are added to help with complexity of the scroll/cursor/search_after APIs

NAME

Search::ElasticDirect - An interface to ElasticSearch API, version independent

VERSION

version 2.5.1

ERRORS

Check for errors the property $obj->{error} . if everything is ok it is FALSE

METHODS

new

Create a new ElasticDirect object and define the connection parameters to the elasticsearch cluster.

https, username, password, CA certificate

Most of the times you will use this one

my $elk = Search::ElasticDirect->new(

  host           => 'SomeServer',
  port           => 9200,
  protocol       => 'https',
  timeout        => 1800,
  keep_alive     => 'yes',
  authentication => 'yes',
  username       => 'Joe',
  password       => 'JoePass',
  certca         => '/etc/elasticsearch/chain.pem',
  verify_SSL     => 'no' );

die $elk->{error} if $elk->{error};

https, CA certificate, server key, server certificate

If you are using the SearchGuard security plugin and you need node admin key/cert without username and password to perform admin tasks e.g. to delete an immutable index

my $elk = Search::ElasticDirect->new(

  host           => 'SomeServer',
  port           => 9200,
  protocol       => 'https',
  timeout        => 1800,
  keep_alive     => 'yes',
  authentication => 'no',
  certca         => '/etc/elasticsearch/chain.pem',
  cert           => '/etc/elasticsearch/SomeServer.pem',
  key            => '/etc/elasticsearch/SomeServer.key',  
  verify_SSL     =>'no' );

die $elk->{error} if $elk->{error};

https, username, password

Without certificate

my $elk = Search::ElasticDirect->new(

  host           => 'SomeServer',
  port           => 9200,
  protocol       => 'https',
  timeout        => 1800,
  keep_alive     => 'yes',
  authentication => 'yes',
  username       => 'Joe',
  password       => 'JoePass' );

die $elk->{error} if $elk->{error};

http, username, password

Plain text with authentication

my $elk = Search::ElasticDirect->new(

  host           => 'SomeServer',
  port           => 9200,
  protocol       => 'http',
  timeout        => 1800,
  keep_alive     => 'yes',
  authentication => 'yes',
  username       => 'Joe',
  password       => 'SomePass' );

die $elk->{error} if $elk->{error};

http

Plain text without authentication

my $elk = Search::ElasticDirect->new(

  host           => 'SomeServer',
  port           => 9200,
  protocol       => 'http',
  timeout        => 1800,
  keep_alive     => 'yes',
  authentication => 'no' );

die $elk->{error} if $elk->{error};

ActiveMaster

If you have schedule your script at multiple servers for high availability but it must run once at active master server

my  $obj = Search::ElasticDirect->new(...);
die $elk->{error} if $elk->{error};

die "Allowed to run only at active master\n" unless $elk->ActiveMaster;
die $elk->{error} if $elk->{error};

direct

Pass through the request to ElasticSeach API transparent without much validation, so you must know the correct syntax and parameters of the API/path you are calling

You can receive the result at the format data, yaml, json, json_pretty, txt

The data format returns a Perl data structure e.g. hash or array, while the other are text formatted as defined

Examples

Health as yaml

my $reply = $elk->direct(method=>'GET', path=>'_cat/health', param=>'filter_path=status', format=>'yaml');
die $elk->{error} if $elk->{error};
say $reply

Health as human readable json

$elk->direct(method=>'GET', path=>'_cat/health', param=>'filter_path=status', format=>'json_pretty');

Health as data; no filter_path is required with data

$reply = $elk->direct(method=>'GET', path=>'_cat/health', format=>'data');
say $reply->[0]->{status};

Indicies sorted

$reply = $elk->direct(method=>'GET', path=>'_cat/indices', param=>'h=index,health,docs.count,store.size&bytes=g&s=index:asc', format=>'data');
die $elk->{error} if $elk->{error};

foreach ( @{ $reply } ) {
say "$_->{index}, $_->{health}, $_->{'docs.count'}, $_->{'store.size'}"
}

DSL query

$reply    = $elk->direct(
method    => 'GET',
path      => 'foo/_search',
format    => 'data',
send_data => '{"size": 2, "query": {"range": {"@timestamp": {"from": "1970-01-01T00:00:00.000Z", "to": "2036-01-03T23:59:59.999Z"}}}}'
);

die $elk->{error} if $elk->{error};

foreach ( @{ $reply->{hits}->{hits} } ) {
say say "$_->{_source}->{'@timestamp'} , $_->{_source}->{id}"
}

SQL query

format can be here data, txt, csv, yaml, json, json_pretty, cbor, smile

$reply = $elk->direct(
method    => 'POST',
path      => '_sql',
format    => 'data',
send_data => "{\"query\": \"SELECT * FROM foo WHERE id >= '01' ORDER BY id DESC LIMIT 100\"}"
);

die $elk->{error} if $elk->{error};

if ($elk->{format} eq 'data') {

  foreach ( @{$reply->{rows}} ) {
  say join ' , ', @{$_}
  }
}
else {
say $reply
}

DeepPagination_scroll

Retrieve massive amount of results using the scroll API

documents here is how many records to fetch per iteration

format can be data, yaml, json or json_pretty

my $iterator = $elk->DeepPagination_scroll(
index     => 'foo',
documents => 9000,
format    => 'data',
send_data => '{"query": {"range": {"@timestamp": {"from": "1970-01-01T00:00:00.000Z", "to": "2036-01-03T23:59:59.999Z"}}}}',
scroll    => '10m'
);

die $elk->{error} if $elk->{error};

while (my $Documents = $iterator->()) {

  if ($elk->{format} eq 'data') {

    foreach my $doc (@{$Documents}) {
  
      foreach my $key (sort keys %{$doc->{_source}}) {
      say "$key : $doc->{_source}->{$key}";
      }
    }
  }
  else {
  say $Documents
  }
}

DeepPagination_search_after

Retrieve massive amount of results using the search_after API

If you choose to use the PIT (point in time) it will created automatic for you

documents here is how many records to fetch per iteration

At sort, define as less fields you can that produce a unique combination. The fields must be sortable reasonable.

my $iterator = $elk->DeepPagination_search_after(
index     => 'foo',
PIT       => 'yes',
keep_alive=> '10m',
documents => 9000,
send_data => '{ "query": {"range": {"@timestamp": {"from": "1970-01-01T00:00:00.000Z", "to": "2036-01-03T23:59:59.999Z"}}} }',
sort      => [ {'@timestamp'=> {order=> 'asc', format=> "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'"}} , {id=>'asc'} ],
);

while (my $Documents = $iterator->()) {

  foreach my $doc (@{$Documents}) {

    foreach my $key (sort keys %{$doc->{_source}}) {
    say "$key : $doc->{_source}->{$key}"
    }
  }
}

DeepPagination_sql

Retrieve massive amount of results using the sql API

documents here is how many records to fetch per iteration

format can be data, csv, txt, yaml, json, json_pretty, cbor, smile

my $iterator = $elk->DeepPagination_sql(
documents => 100,
format    => 'data',
send_data => '{ "query": "SELECT * FROM foo ORDER BY id DESC" }'
);

die $elk->{error} if $elk->{error};

while (my $Documents = $iterator->()) {

  if ($elk->{format} eq 'data') {
  use Data::Dumper; say Dumper $Documents
  }
  else {
  print $Documents
  }
}

SEE OTHER

Search::Elasticsearch The official client for Elasticsearch

Dancer2::Plugin::ElasticSearch ElasticSearch wrapper for Dancer

SUPPORT

Any ideas or contributors are welcome

AUTHOR

George Bouras <george.mpouras@yandex.com>

COPYRIGHT AND LICENSE

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.

AUTHOR

George Bouras <george.mpouras@yandex.com>

COPYRIGHT AND LICENSE

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.

To install Search::ElasticDirect, copy and paste the appropriate command in to your terminal.

cpanm

cpanm Search::ElasticDirect

CPAN shell

perl -MCPAN -e shell
install Search::ElasticDirect

For more information on module installation, please visit the detailed CPAN module installation guide.

	Global
`s`	Focus search bar
`?`	Bring up this help dialog

	GitHub
`g` `p`	Go to pull requests
`g` `i`	go to github issues (only if github is preferred repository)

	POD
`g` `a`	Go to author
`g` `c`	Go to changes
`g` `i`	Go to issues
`g` `d`	Go to dist
`g` `r`	Go to repository/SCM
`g` `s`	Go to source
`g` `b`	Go to file browse

	Search terms
module: (e.g. module:Plugin)
distribution: (e.g. distribution:Dancer auth)
author: (e.g. author:SONGMU Redis)
version: (e.g. version:1.00)

NAME

VERSION

SYNOPSIS

DESCRIPTION

NAME

VERSION

ERRORS

METHODS

new

https, username, password, CA certificate

https, CA certificate, server key, server certificate

https, username, password

http, username, password

http

ActiveMaster

direct

Health as yaml

Health as human readable json

Health as data; no filter_path is required with data

Indicies sorted

DSL query

SQL query

DeepPagination_scroll

DeepPagination_search_after

DeepPagination_sql

SEE OTHER

SUPPORT

AUTHOR

COPYRIGHT AND LICENSE

AUTHOR

COPYRIGHT AND LICENSE

Module Install Instructions