NAME
Parallel::Queue - fork subref's N-way parallel as static list or as generated by an object.
SYNOPSIS
# example queue:
# only squish files larger than 8KB in size. figure
# that the system can handle four copies of squish
# running at the same time without them interfering
# with one another.
my @queue = map { -s > 8192 ? sub{ squish $_ } : () } @filz;
# simplest case: use the module and pass in
# the count and list of coderefs.
use Parallel::Queue;
my @remaining = runqueue 4, @queue;
die "Incomplete jobs" if @remaining;
# an object with method "next_job" will be called
# as $iterator->next_job until it returns false. the
# return values are dispatched via $sub->();
# $iter can be an object or class, including
# __PACKAGE__ if the package sets itself up to
# handle the paralell jobs.
my $iter = Foo->new( @job_parmz );
runqueue 4 => $iter;
$pkg->configure( @job_parmz );
runqueue 8 => $pkg;
# export allows changing the exported sub name.
# "export=" allows not exporting it (which then
# requires calling Parallel::Queue::runqueue ...
use Parallel::Queue qw( export=handle_queue );
my @remaining = handle_queue 4, @queue;
# "fork" is the normal state.
# "nofork" or a zero count avoid forking.
# detecting the perl debugger (via $^P) will
# defaults "fork" to false ("nofork" mode).
# forking in the debugger can be turned on
# with an explicit fork (which includs a
# warning for lack of $DB::DEBUG_TTY).
#!/usr/bin/perl -d
use Parallel::Queue; # defaults to nofork mode.
use Parallel::Queue qw( nofork ); # ditto
use Parallel::Queue qw( fork ); # forks, even in the debugger.
# "debug" turns on nofork and verbose.
# these produce identical results.
use Parallel::Queue qw( debug );
use Parallel::Queue qw( nofork verbose );
# finish forces execution to continue even if
# there is an error in one job. this will finish
# the cleanups even if one of them fails.
use Parallel::Queue qw( finish );
my @cleanupz = ... ;
runqueue $nway, @cleanupz;
# "configure" is a more descriptive alias for the
# import sub.
Parallel::Queue->configure( debug=0 finish=1 );DESCRIPTION
Arguments to use (or configure).
The finish, debug, and verbose arguments default to true. This means that turning them on does not require an equals sign and value. Turning them off requries an equal sign and zero.
The export option defaults to false: using it without a value avoids exporting the runqueue subroune into the caller.
- finish (default finish=0)
-
This causes the queue to finsih even if there are non-zero exits on the way. Exits will be logged but will not stop the queue.
- export=my_name (default export=runqueue)
-
By default Parallel::Queue exports "runqueue", this can be changed with the "export" arguments. In this case call it "my_name" and use it to run the queue with two parallel processes:
use Parallel::Queue qw( export=run_these ); my @queue = ...; my @un_executed = run_these 2, @queue;The name can be any valid Perl sub name.
Using an empty name avoids exporting anything and requires using the fully qualified subname (Parallel::Query::runqueu) to run the queue.
- verbose
-
This outputs a line with the process id each time a job is dispatched or reaped.
- debug
-
Turned on by default if the perl debugger is in use (via $^P), this avoids forking and simply dispatches the jobs one by one. This helps debug dispatched jobs where handling forks in the Perl debugger can be problematic.
Debug can be turned off via debug=0 with the use or confgure. If this is turned off with the debugger running then be prepared to supply the tty's (see also Debugging forks below).
KNOWN ISSUES
- Non-numeric count arguments.
-
The runqueue sub uses Scalar::Util::looks_like_number validate the count. This may cause problems for objects which don't look like numbers.
SEE ALSO
COPYRIGHT
This code is released under the same terms as Perl-5.20 or any later version of Perl.
AUTHOR
Steven Lembark <lembark@wrkhors.com>