NAME

Datahub::Factory::Importer::MSK - Import data from the MSK OAI-PMH endpoint

SYNOPSIS

use Datahub::Factory::Importer::MSK;
use Data::Dumper qw(Dumper);

my $oai = Datahub::Factory::Importer::MSK->new(
    url                    => 'https://endpoint.msk.be/oai',
    metadataPrefix         => 'oai_lido',
    set                    => '2011',
    pid_username           => 'datahub',
    pid_password           => 'datahub',
    pid_rcf_container_name => 'datahub',
);

$oai->importer->each(sub {
    my $item = shift;
    print Dumper($item);
});

DESCRIPTION

Datahub::Factory::Importer::MSK imports data from the MSK OAI-PMH endpoint. By default it uses the ListRecords verb to return all records using the oai_lido format. It is possible to only return records from a single Set or those created, modified or deleted between two dates (from and until).

It automatically deals with resumptionTokens, so client code does not have to implement paging.

To support PIDs, it uses Rackspace Cloud Files to fetch PID CSV's and convert them to temporary sqlite tables.

Provide pid_username, pid_password and pid_rcf_container_name.

PARAMETERS

Only the endpoint parameter is required.

endpoint

URL of the OAI endpoint.

handler( sub {} | $object | 'NAME' | '+NAME' ) Handler to transform each record from XML DOM (XML::LibXML::Element) into Perl hash.

Handlers can be provided as function reference, an instance of a Perl package that implements 'parse', or by a package NAME. Package names should be prepended by + or prefixed with Catmandu::Importer::OAI::Parser. E.g foobar will create a Catmandu::Importer::OAI::Parser::foobar instance. By default the handler Catmandu::Importer::OAI::Parser::oai_dc is used for metadataPrefix oai_dc, Catmandu::Importer::OAI::Parser::marcxml for marcxml, Catmandu::Importer::OAI::Parser::mods for mods, Catmandu::Importer::OAI::Parser::Lido for Lido and Catmandu::Importer::OAI::Parser::struct for other formats. In addition there is Catmandu::Importer::OAI::Parser::raw to return the XML as it is.

metadata_prefix

Any metadata prefix the endpoint supports. Defaults to oai_lido.

set

Optionally, a set to get records from.

from

Optionally, a must_be_older_than date.

until

Optionally, a must_be_younger_than date.

pid_username

Provide your Rackspace Cloud Files username.

pid_password

Provide your Rackspace Cloud Files api key.

pid_rcf_container_name

Provide the container name that holds the PID CSV's.

ATTRIBUTES

importer

A Importer that can be used in your script.

AUTHOR

Pieter De Praetere <pieter at packed.be >

COPYRIGHT

Copyright 2017- PACKED vzw

LICENSE

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

SEE ALSO

Datahub::Factory Catmandu