NAME

MyCPAN::Indexer::Tutorial - How the backpan_indexer.pl pieces fit together

DESCRIPTION

The MyCPAN::Indexer system lets you plug in different components to control major portions of the process of examining Perl distributions and collating the results. It's up to each component to obey its interface and do that parts the other components expect it to do. The idea is to decouple some of these bits as much as possible.

As backpan_indexer.pl does its work, it stores information about its components in an anonymous hash called notes. The different components have access to this hash. (To Do: this is some pretty bad design smell, but that's how it is right now).

Specific implementations will impose other requirements not listed in this tutorial.

The Application

The application is the bit that you write when you want to do something very specialized with a different process. The application object controls the environment, setting up the configuration, and other application-level type things.

See MyCPAN::Indexer::App::BackPAN, the module version, and backpan_indexer.pl, the script version.

The Coordinator

The coordinator is just a way for the components to talk to each other. The application starts up, creates a coordinator object, and stores it. The application gives a reference to the coordinator to every component.

When the application creates components, it tells each about the coordinator. Each component can talk to the coordinator to get to parts of the application it doesn't directly know about. Each component tells the coordinator about itself so the coordinator can talk to any component.

The coordinator also maintains the "notes", which are arbitrary bits of information that components use to pass information around. The notes are like a scratch pad where components can leave information for other components.

See MyCPAN::Indexer::Coordinator.

Components

The Indexer class

Most of the work to examine a Perl distribution is in MyCPAN::Indexer. When it gets down to it, everything MyCPAN knows about Perl distributions is in there. It has a run() method which handles the examination. It kicks off examine, which figures out what to do by getting a list of steps from examine_dist_steps.

This technique is common throughout MyCPAN::Indexer. One method returns a list of methods to run. This way, a subclass can control the process by overriding the method that returns the steps.

The basic class is MyCPAN::Indexer, but MyCPAN::Indexer::TestCensus is an example of another indexing class.

Implements:

get_indexer()

Creates in notes:

indexer_callback - a sub reference that wraps its run routine

Expects in notes:

nothing

The Queue class

The Queue class is responsible for getting the list of distributions to process.

backpan_indexer.pl calls get_queue and passes it a ConfigReader::Simple object. get_queue does whatever it needs to do, then returns an array reference of file paths to process. Each path should represent a single Perl distribution.

Implements:

get_queue()

Creates in notes:

queue - a reference to the array reference returned by get_queue.

Expects in notes:

nothing

To Do: The Queue class should really be an iterator of some sort. Instead of returning an array (which it can't change), return an iterator.

The Worker class

The Worker class returns the anonymous subroutine that the interface class calls for each of its cycles. Inside that code reference, do the actual indexing work, including saving the results. backpan_indexer.pl calls get_task with a reference to its notes hash.

Implements:

get_task()

Creates in notes

child_task - a reference to the code reference returned by get_task.

Expects in notes

nothing

To Do: There should be a storage class which the worker class hands the results to.

The Reporter class

The Reporter class implements the bits to store the result of the Worker class. backpan_indexer.pl calls get_reporter with a reference to its notes hash.

Implements:

get_reporter( $info )

Creates in notes:

reporter - the code ref to handle storing the information

Expects in notes:

nothing

Expects in config:

nothing

The Dispatcher class

The Dispatcher class implements the bits to hand out work to the worker class. The Interface class, discussed next, repeatedly calls the interface_callback code ref the Dispatcher class provides.

Implements:

get_dispatcher()

Creates in notes

dispatcher - the dispatcher object, with start and finish methods
interface_callback - a code ref to call repeatedly in the Interface class

Expects in notes

child_task - the code ref that handles indexing a single dist
queue      - the array ref of dist paths

The Collator class

After the Dispatcher class finishes its queue of work, the Collator class comes in and does something with the collection of reports.

Implements:

get_collator()

Creates in notes

collator - a subroutine reference

Expects in notes

nothing

The Interface class

The Interface class really has two jobs. It makes the live reporting interface while backpan_indexer.pl runs, at it repeatedly calls the dispatcher to start new work.

Implements:

do_interface()

Creates in notes:

nothing

Expects in notes

interface_callback - a code ref to call repeatedly in the Interface class

SEE ALSO

MyCPAN::Indexer

SOURCE AVAILABILITY

This code is in Github:

git://github.com/briandfoy/mycpan-indexer.git

AUTHOR

brian d foy, <bdfoy@cpan.org>

COPYRIGHT AND LICENSE

Copyright © 2008-2018, brian d foy <bdfoy@cpan.org>. All rights reserved.

You may redistribute this under the terms of the Artistic License 2.0.

cpanm MyCPAN::Indexer

perl -MCPAN -e shell
install MyCPAN::Indexer