NAME
Datahub::Factory - A conveyor belt which transports data from a data source to a data sink.
SYNOPSIS
dhconveyor command OPTIONS
DESCRIPTION
Datahub::Factory is a command line conveyor belt which automates three tasks:
- Data is fetched automatically from a local or remote data source.
- Data is converted to an exchange format.
- The output is pushed to a data sink.
Datahub::Factory fetches data from several sources as specified by the Importer settings, executes a Catmandu::Fix and sends it to a data sink, set via an Exporter. Several importer and exporter modules are provided, but developers can extend the functionality with their own modules.
Datahub::Factory contains Log4perl support.
CONFIGURATION
Command line options
All commands share the following switches:
- --log_level
-
Set the log_level. Takes a numeric parameter. Supported levels are: 1 (WARN), 2 (INFO), 3 (DEBUG). WARN (1) is the default.
- --log_output
-
Selects an output for the log messages. By default, it will send them to STDERR (pass STDERR as parameter), but STDOUT (STDOUT) and a log file.
- --verbose
-
Set verbosity. Invoking the command with the --verbose, -v flag will render verbose output to the terminal.
COMMANDS
help COMMAND
Documentation about command line options.
transport OPTIONS
Fetch data from a local or remote source, convert it to an exchange format and export the data.
transport OPTIONS
Fetch data from a local source, and push it to an enterprise search engine via a bulk API.
API
Datahub::Factory uses a plugin-based architecture, making it easy to extend with new functionality.
New commands can be added by creating a Perl module that contains a `command_name.pm` file in the `lib/Datahub/Factory/Command` path. Datahub::Factory uses the Datahub::Factory::Command namespace and leverages App::Cmd internally.
New Datahub::Factory::Importer, Datahub::Factory::Exporter and Datahub::Factory::Fixer plugins can be added in the same way.
AUTHORS
Matthias Vandermaesen <matthias.vandermaesen@vlaamsekunstcollectie.be> Pieter De Praetere <pieter@packed.be>
COPYRIGHT AND LICENSE
This software is copyright (c) 2016 by PACKED, vzw, Vlaamse Kunstcollectie, vzw.
This is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License, Version 3, June 2007.