NAME

DynGig::Util::MapReduce - A Map Reduce Job Launcher

SYNOPOSIS

use DynGig::Util::MapReduce;

my $batch = sub { ..  };
my $map = sub { .. };
my $reduce = sub { .. };

my $job = DynGig::Util::MapReduce->new
(
    name => 'foo',
    batch => $batch,
    map => $map,
    reduce => $reduce,
);

my %batch_param = ( .. );
my %map_param = ( .. );
my %reduce_param = ( .. );
my %context = ( .. );

$job->run
(
    context => \%context,
    batch => \%batch_param,
    map => \%map_param,
    reduce => \%reduce_param,
);

my $result = $job->result();

DESCRIPTION

Map/Reduce is an approach that collects data in parallel then processes data in serial. A Map/Reduce job has 4 components/steps. Each component is to be defined by the user.

Batch : divide the targets into batches.
Map   : create threads to collect data from/for each batch in parallel.
Sort  : aggregate collected data by status.
Reduce: (optional) process aggregated data serially.

run( batch => HASH, map => HASH, reduce => HASH, context => HASH )

result()

Returns the result of the run as a HASH reference or undef if invoked before run.

context()

Returns the context of the run as a HASH reference or undef if invoked before run.

name()

Returns the name of the job.

SEE ALSO

threads, Thread::Queue, and YAML::XS for data serialization.

NOTE

See DynGig::Util