NAME
Proc::tored::Pool - managed work pool with Proc::tored and Parallel::ForkManager
VERSION
version 0.07
SYNOPSIS
use Proc::tored::Pool;
# Create a worker pool service
my $pool = pool 'thing-doer', in '/var/run', capacity 10,
on success, call {
my ($me, $id, @results) = @_;
print "thing $id complete: @results";
},
on failure, call {
my ($me, $id, $message) = @_;
warn "thing $id failed: $message";
};
# Do things with the pool
run {
my ($thing_id, $thing) = get_next_thing();
process { do_things($thing) } $pool, $thing_id;
} $pool;
# Control the pool as a Proc::tored service
pause $pool;
resume $pool;
stop $pool;
zap $pool, 15 or die "timed out after 15 seconds waiting for pool to stop";
start $pool;DESCRIPTION
Provides a simple and fast interfact to build and manage a pool of forked worker processes. The process is controlled using a pidfile and touch file.
EXPORTED SUBROUTINES
As a Proc::tored::Pool is a Proc::tored service, it by default exports the same functions as "EXPORTED SUBROUTINES" in Proc::tored.
In addition, the following subroutines are exported by default.
pool
Creates the pool (an instance of Proc::tored::Pool::Manager). Requires a $name as its first argument.
my $pool = pool 'the-proletariat', ...;capacity
Sets the max number of forked worker processes to be permitted at any given time.
my $pool = pool 'the-proletariat', capacity 16, ...;on
Builds an event callback with one of "assignment", "success", or "failure".
my $pool = pool 'the-proletariat', capacity 16,
on success, call { ... };call
Defines the code to be called by an event callback. See "on".
pending
Returns the number of tasks that have been assigned to worker processes but have not yet completed.
process
Sends a task (a CODE ref) to the pool, optionally specifying a task id to identify the results in callbacks. The return value of the supplied code ref is passed as is to the "success" callback (if supplied).
process { seize_the_means_of_production() } $pool;
process { seize_the_means_of_production() } $pool, $task_id;sync
For situations in which a task or tasks must be completed before program execution can continue, sync may be used to block until all pending tasks have completed. After calling sync, there will be no pending tasks and all callbacks for previously submitted tasks will have been called.
process { seize_the_means_of_production() } $pool;
sync $pool;EVENTS
assignment
Triggered immediately after a task is assigned to a worker process. Receives the pool object and the task id (if provided when calling "pool").
my $pool = pool 'thing-doer', ...,
on assignment, call {
my ($self, $task_id) = @_;
$assigned{$task_id} = 1;
};
process { do_things() } $pool, $task_id;success
Triggered after the completion of a task. Receives the pool object, task id (if provided when calling "pool"), and the return value of the code block.
my $pool = pool 'thing-doer', ...,
on success, call {
my ($self, $task_id, @results) = @_;
...
};
process { do_things() } $pool, $task_id;If "process" performs an exec, no data is returned from the worker process (because exec never returns). In this case, if the process had a zero exit status, the success callback is triggered with the results value of "zero but true". A non-zero exit status is handled by failure as in the general case.
failure
Triggered if the code block dies or the forked worker exits abnormally. Recieves the pool object, task id (if provided when calling "pool"), and the error message generated by the code ref.
my $pool = pool 'thing-doer', ...,
on failure, call {
my ($self, $task_id, $error) = @_;
warn "Error executing task $task_id: $error";
};
process { do_things() } $pool, $task_id;BUGS AND LIMITATIONS
Proc::tored
Warnings and limitations for Proc::tored also apply to this module, especially the notes regarding service control from within a running service which also apply to code executing in forked worker processes. See "BUGS AND LIMITATIONS" in Proc::tored for details.
Parallel::ForkManager
Warnings and limitations for Parallel::ForkManager also apply to this module, including the injunction against using two pools simultaneously from the same process. See "BUGS AND LIMITATIONS" in Parallel::ForkManager and "SECURITY" in Parallel::ForkManager for details.
SEE ALSO
AUTHOR
Jeff Ober <jeffober@gmail.com>
COPYRIGHT AND LICENSE
This software is copyright (c) 2017 by Jeff Ober.
This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.