NAME
Piper::Instance - An initialized pipeline segment for the Piper system
ATTRIBUTES
batch_size
The number of items to process at a time for this segment.
If not set, inherits the batch_size of any existing parent(s). If the segment has no parents, or if none of its parents have a batch_size defined, the default batch_size will be used. The default is 200, but this can be configured at import of Piper.
To clear a previously-set batch_size, simply set it to undef or use the clear_batch_size method.
$segment->batch_size(undef);
$segment->clear_batch_size;children
For container instances (made from Piper objects, not Piper::Process objects), the children attribute holds an arrayref of the contained instance objects.
debug
Debug level for this segment. When accessing, inherits the debug level of any existing parent(s) if not explicitly set for this segment. The default level is 0, but can be globally overridden with the environment variable PIPER_DEBUG.
To clear a previously-set debug level for a segment, simply set it to undef or use the clear_debug method.
$segment->debug(undef);
$segment->clear_debug;enabled
A boolean indicating that the segment is enabled and can accept items for processing. Inherits this attribute from any existing parent(s) with a default of true.
To clear a previously-set enabled attribute, simply set it to undef or use the clear_enabled method.
$segment->enabled(undef);
$segment->clear_enabled;main
Holds a reference to the outermost container instance for the pipeline.
parent
Unless this segment is the outermost container (main), this attribute holds a reference to the segment's immediate container.
path
The full path to this segment, built as the concatenation of all the parent(s) labels and the segment's label, joined by /. Piper::Instance objects stringify to this attribute.
verbose
Verbosity level for this segment. When accessing, inherits verbosity level of any existing parent(s) if not explicitly set for this segment.
To clear a previously-set verbosity level for a segment, simply set it to undef or use the clear_verbose method.
$segment->verbose(undef);
$segment->clear_verbose;METHODS
Methods marked with a (*) should only be called from the outermost instance.
clear_batch_size
clear_debug
clear_enabled
clear_verbose
Methods for clearing the corresponding attribute.
has_children
A boolean indicating whether the instance has any children (contained instances). Will be true for all segments initialized from a Piper object and false for all segments initialized from a Piper::Process object.
has_parent
A boolean indicating whether the instance has a parent (container instance). Will be true for all segments except the outermost segment (main).
has_pending
Returns a boolean indicating whether there are any items that are queued at some level of the segment but have not completed processing.
*dequeue([$num])
Remove at most $num (default 1) processed items from the end of the segment.
*enqueue(@data)
Queue @data for processing by the pipeline.
find_segment($location)
Find and return the segment instance according to <$location>, which can be a label or a path-like hierarchy of labels.
For example, in the following pipeline, a few possible $location values include a, subpipe/b, or main/subpipe/c.
my $pipe = Piper->new(
{ label => 'main' },
subpipe => Piper->new(
a => sub { ... },
b => sub { ... },
c => sub { ... },
),
)->init;If a label is unique within the pipeline, no path is required. For non-unique labels, searches are performed in a nearest-neighbor, depth-first manner.
For example, in the following pipeline, searching for processA from processB would find main/pipeA/processA, not main/processA. So to reach main/processA from processB, the appropriate search would be for main/processA.
my $pipe = Piper->new(
{ label => 'main' },
pipeA => Piper->new(
processA => sub { ... },
processB => sub { ... },
),
processA => sub { ... },
);*flush
Process batches until there are no more items pending.
*is_exhausted
Returns a boolean indicating whether there are any items left to process or dequeue.
*isnt_exhausted
Returns the opposite of is_exhausted.
next_segment
Returns the next adjacent segment from the calling segment. Returns undef for the outermost container.
pending
Returns the number of items that are queued at some level of the segment but have not completed processing.
*prepare([$num])
Process batches while data is still pending until at least $num (default 1) items are ready for dequeue.
ready
Returns the number of items that have finished processing and are ready for dequeue from the segment.
FLOW CONTROL METHODS
These methods are available for use within process handler subroutines (see Piper::Process).
eject(@data)
If the segment has a parent, send @data to the drain of its parent. Otherwise, enqueues @data to the segment's drain.
emit(@data)
Send @data to the next segment in the pipeline. If the segment is the last in the pipeline, emits to the drain, making the @data ready for dequeue.
inject(@data)
If the segment has a parent, enqueues @data to its parent. Otherwise, enqueues <@data> to itself.
injectAfter($location, @data)
Send @data to the segment after the specified $location. See find_segment for a detailed description of $location.
injectAt($location, @data)
Send @data to the segment at the specified $location. See find_segment for a detailed description of $location.
recycle(@data)
Re-queue @data to the top of the current segment in an order such that dequeue(1) would subsequently return $data[0] and so forth.
LOGGING AND DEBUGGING METHODS
See Piper::Logger for detailed descriptions.
INFO($message, [@items])
Prints an informational $message to STDERR if either the debug or verbosity level for the segment is > 0.
DEBUG($message, [@items])
Prints a debug $message to STDERR if the debug level for the segment is > 0.
WARN($message, [@items])
Issues a warning with $message via Carp::carp.
ERROR($message, [@items])
Throws an error with $message via Carp::croak.
UTILITY ATTRIBUTES
None of these should be directly accessed. Documented for contributors and source-code readers.
args
The arguments passed to the init method of Piper.
directory
A hashref of the segment's children, keyed by their labels. Used by find_segment.
drain
A reference to the location where the segment's processed items are emitted.
follower
A hashref of children paths to the child's next adjacent segment. Used by next_segment.
logger
A reference to the logger for the pipeline. Handles "LOGGING AND DEBUGGING" methods.
queue
A reference to the location where data is queued for processing by this segment.
segment
The Piper or Piper::Process object from which the instance segment was created.
UTILITY METHODS
None of these should be directly accessed. Documented for contributors and source-code readers.
descendant($path, $referrer)
Returns a child segment if its path ends with $path. Does not search children with a path of $referrer, as it was presumably already searched by a previous iteration of the search. Used by find_segment.
pressure
An integer metric for the "fullness" of the pending queue. For handler instances (initialized from Piper::Process objects), it is the percentage of pending items vs the batch size of the segment. For container instances (initialized from Piper objects), is is the maximum pressure of the contained instances. Used by process_batch for choosing which segment to process.
process_batch
Chooses the "best" segment for processing, and processes a batch for that segment.
It first attempts to choose the full-batch segment (pending >= batch_size) closest to the end of the pipeline. If there are no full-batch segments, it chooses the segment closest to being full.
SEE ALSO
VERSION
version 0.05
AUTHOR
Mary Ehlers <ehlers@cpan.org>
COPYRIGHT AND LICENSE
This software is Copyright (c) 2017 by Mary Ehlers.
This is free software, licensed under:
The Apache License, Version 2.0, January 2004