/ 
Datahub-Factory-0.11
River stage one • 4 direct dependents • 4 total dependents
/ Datahub::Factory::Command::transport

NAME

Datahub::Factory::Command::transport - Implements the 'transport' command.

DESCRIPTION

This command allows datamanagers to (a) fetch data from a (local) source (b) transform the data to LIDO using a fix (c) upload the LIDO transformed data to a Datahub instance.

COMMAND LINE INTERFACE

Options

--importer NAME

The importer which fetches data from a Collection Registration system. Currently only "Adlib" and "TMS" are supported options. All --oimport arguments are tied to the specific importer used.

--fixes PATH

The path to the Catmandu Fix files to transform the data.

--exporter NAME

The exporter that will do something with your data. It is possible to print to STDOUT in a specific format ("YAML" and "LIDO" are supported) or to export to a Datahub instance. All --oexport arguments are tied to the specific exporter used.

--oimport file_name=PATH

The path to a flat file containing data. This option is only relevant when the input is an Adlib XML export file.

--oimport db_user=VALUE

The database user. This option is only relevant when the input is an TMS database.

--oimport db_passowrd=VALUE

The database user password. This option is only relevant when the input is an TMS database.

--oimport db_name=VALUE

The database name. This option is only relevant when the input is an TMS database.

--oimport db_host=VALUE

The database host. This option is only relevant when the input is an TMS database.

--oexport datahub_url=VALUE

The URL to the datahub instance. This should be a FQDN ie. http://datahub.lan/

--oexport oauth_client_id=VALUE

The client public ID. Used for OAuth authentication of the Datahub endpoint.

--oexport oauth_client_secret=VALUE

The client secret passphrase. Used for OAuth authentication of the Datahub endpoint.

--oexport oauth_username=VALUE

The username of the Datahub user. Used for OAuth authentication of the Datahub endpoint.

--oexport oauth_password=VALUE

The password of the Datahub user. Used for OAuth authentication of the Datahub endpoint.

Pipeline configuration file

The pipeline configuration file is in the INI format and its location is provided to the application using the --pipeline switch.

The file is broadly divided in two parts: the first (shortest) part configures the pipeline itself and sets the plugins to use for the import, fix and export actions. The second part sets options specific for the used plugins.

Pipeline configuration

This part has three sections: [Importer], [Fixer] and [Exporter]. Every section has just one option: plugin. Set this to the plugin you want to use for every action.

All current supported plugins are in the Importer and Exporter folders. For the [Fixer], only the Fix plugin is supported.

Supported Importer plugins:

TMS
Adlib
OAI

Supported Exporter plugins:

Datahub
LIDO
YAML

Plugin configuration

All plugins have their own configuration options in sections called [plugin_type_name] where type can be importer, exporter or fixer and name is the name of the plugin (see above).

For a list of supported and required options, see the plugin documentation.

The [plugin_fixer_Fix] has two options: id_path and file_name.

The id_path option contains the path (in Fix syntax) of the identifier of each record in your data after the fix has been applied, but before it is submitted to the Exporter. It is used for reporting and logging.

file_name points to the location of your Fix file.

Example configuration file

[Importer]
plugin = Adlib

[Fixer]
plugin = Fix
id_path = 'administrativeMetadata.recordWrap.recordID.0._'

[Exporter]
plugin = Datahub

[plugin_importer_Adlib]
file_name = '/tmp/adlib.xml'
data_path = 'recordList.record.*'

[plugin_fixer_Fix]
file_name = '/tmp/msk.fix'
id_path = ''

[plugin_exporter_Datahub]
datahub_url = https://my.thedatahub.io
datahub_format = LIDO
oauth_client_id = datahub
oauth_client_secret = datahub
oauth_username = datahub
oauth_password = datahub

AUTHORS

Pieter De Praetere <pieter@packed.be>
Matthias Vandermaesen <matthias.vandermaesen@vlaamsekunstcollectie.be>

COPYRIGHT

Copyright 2016 - PACKED vzw, Vlaamse Kunstcollectie vzw

LICENSE

This library is free software; you can redistribute it and/or modify it under the terms of the GPLv3.