NAME

recs-decollate

recs-decollate --help-all

Help from: --help-basic:
Usage: recs-decollate <args> [<files>]
   Decollate records of input (or records from <files>) into output records.

Arguments:
   --dldeaggregator ...                Specify a domain language aggregate. See
                                       "Domain Language Integration" below.
   --deaggregator|-d <deaggregators>   Colon separated list of aggregate
                                       field specifiers. See "Deaggregates"
                                       section below.
   --list-deaggregators                Bail and output a list of deaggregators.
   --show-deaggregator <deaggregator>  Bail and output this deaggregator's
                                       detailed usage.
   --filename-key|fk <keyspec>         Add a key with the source filename (if no
                                       filename is applicable will put NONE)

  Help Options:
      --help-all             Output all help for this script
      --help                 This help screen
      --help-deaggregators   List the deaggregators
      --help-domainlanguage  Help on the recs domain language, a [very
                             complicated] way of specifying valuations (which
                             act like keys) or aggregators

Deaggregates:
   Deaggregates are specified as <deaggregator>[,<arguments>]. See --list-
   deaggregators for a list of available deaggregators.

   In general, key name arguments to deaggregators may be key specs, but not
   key groups

Domain Lanuage Integration:
   The normal mechanism for specifying keys and aggregators allows one to
   concisely instantiate the objects that back them in the platform and is
   certainly the easiest way to use recs. The record stream domain language
   allows the creation of these objects in a programmatic way, with neither the
   syntactic issues of the normal way nor its guiding hand.

   The domain language is itself just PERL with a collection of library
   functions for creating platform objects included. Your favorite aggregators
   are all here with constructors matching their normal token. For convenience
   of e.g. last, aggregators are also included with a prefixed underscore.

   Below you can find documentation on all the "built in" functions. Most
   aggregators and deaggregators should be present with arguments comparable to
   their normal instantiation arugments, but with keyspec parameters replaced
   with valuations parameters.

   Deaggregates may be specified using the recs domain language. --
   dldeaggregator requires the code evaluate as a deaggregator.

   See --help-domainlanguage for a more complete description of its workings and
   a list of available functions.

   See the examples below for a more gentle introduction.

Examples:
   Split the "hosts" field into individual "host" fields
      recs-decollate --dldeaggregator '_split(hosts,qr/, */,host)'

Help from: --help-deaggregators:
split: split the provided field
unarray: split the provided array
unhash: split the provided hash

Help from: --help-domainlanguage:
DOMAIN LANGUAGE
   The normal mechanism for specifying keys and aggregators allows one to
   concisely instantiate the objects that back them in the platform and is
   certainly the easiest way to use recs. The record stream domain language
   allows the creation of these objects in a programmatic way, with neither the
   syntactic issues of the normal way nor its guiding hand.

   The domain language is itself just PERL with a collection of library
   functions for creating platform objects included. Your favorite aggregators
   are all here with constructors matching their normal token. For convenience
   of e.g. last, aggregators are also included with a prefixed underscore.

   Below you can find documentation on all the "built in" functions. Most
   aggregators and deaggregators should be present with arguments comparable to
   their normal instantiation arugments, but with keyspec parameters replaced
   with valuations parameters.

Special Syntax
   Where one sees a <snippet> argument below, a string scalar is expected,
   however quoting these can get fairly difficult and they can be confused with
   non-<snippet> scalars.

   Example:
     --dla "silly= uconcat(',', snip('{{x}} * 2'))"

   To remedy this, one may use <<CODE>> to inline a snippet which will be
   immediately understood by the typing mechanism as being code. Escaping inside
   this is as single quotes in PERL.

   Example With <<CODE>>
     --dla 'silly= uconcat(",", <<{{x}} * 2>>)'

   Furthermore one may mark variables to be propagated in by prefixing CODE like
   <<var1,var2,var3|CODE>>:
     --dla 'silly= $f=2; uconcat(",", <<f|{{x}} * $f>>)'

Function Library
   ii_agg(<snippet>, <snippet>[, <snippet>])
   ii_aggregator(<snippet>, <snippet>[, <snippet>])
   inject_into_agg(<snippet>, <snippet>[, <snippet>])
   inject_into_aggregator(<snippet>, <snippet>[, <snippet>])
      Take an initial snippet, a combine snippet, and an optional squish snippet
      to produce an ad-hoc aggregator based on inject into. The initial snippet
      produces the aggregate value for an empty collection, then combine takes
      $a representing the aggregate value so far and $r representing the next
      record to add and returns the new aggregate value. Finally, the squish
      snippet takes $a representing the final aggregate value so far and
      produces the final answer for the aggregator.

      Example(s):
         Track count and sum to produce average:
            ii_agg(<<[0, 0]>>, <<[$a->[0] + 1, $a->[1] + {{ct}}]>>, <<$a->[1] / $a->[0]>>)

   for_field(qr/.../, <snippet>)
      Takes a regex and a snippet of code. Creates an aggregator that creates a
      map. Keys in the map correspond to fields chosen by matching the regex
      against the fields from input records. Values in the map are produced by
      aggregators which the snippet must act as a factory for ($f is the field).

      Example(s):
         To aggregate the sums of all the fields beginning with "t"
            for_field(qr/^t/, <<sum($f)>>)

   for_field(qr/.../, qr/.../, <snippet>)
      Takes two regexes and a snippet of code. Creates an aggregator that
      creates a map. Keys in the map correspond to pairs of fields chosen by
      matching the regexes against the fields from input records. Values in the
      map are produced by aggregators which the snippet must act as a factory
      for ($f1 is the first field, $f2 is the second field).

      Example(s):
         To find the covariance of all x-named fields with all y-named fields:
            for_field(qr/^x/, qr/^y/, <<covar($f1, $f2)>>)

   map_reduce_agg(<snippet>, <snippet>[, <snippet>])
   map_reduce_aggregator(<snippet>, <snippet>[, <snippet>])
   mr_agg(<snippet>, <snippet>[, <snippet>])
   mr_aggregator(<snippet>, <snippet>[, <snippet>])
      Take a map snippet, a reduce snippet, and an optional squish snippet to
      produce an ad-hoc aggregator based on map reduce. The map snippet takes $r
      representing a record and returns its mapped value. The reduce snippet
      takes $a and $b representing two mapped values and combines them. Finally,
      the squish snippet takes a mapped value $a representing all the records
      and produces the final answer for the aggregator.

      Example(s):
         Track count and sum to produce average:
            mr_agg(<<[1, {{ct}}]>>, <<[$a->[0] + $b->[0], $a->[1] + $b->[1]]>>, <<$a->[1] / $a->[0]>>)

   rec()
   record()
      A valuation that just returns the entire record.

   snip(snip)
      Takes a snippet and returns both the snippet and the snippet as a
      valuation. Used to distinguished snippets from scalars in cases where it
      matters, e.g. min('{{x}}') interprets it is a keyspec when it was meant to
      be a snippet (and then a valuation), min(snip('{{x}}')) does what is
      intended. This is used internally by <<...>> and in fact <<...>> just
      translates to snip('...').

   subset_agg(<snippet>, <aggregator>)
   subset_aggregator(<snippet>, <aggregator>)
      Takes a snippate to act as a record predicate and an aggregator and
      produces an aggregator that acts as the provided aggregator as run on the
      filtered view.

      Example(s):
          An aggregator that counts the number of records with a time not above 6 seconds:
             subset_agg(<<{{time_ms}} <= 6000>>, ct())

   type_agg(obj)
   type_scalar(obj)
   type_val(obj)
      Force the object into a specific type. Can be used to force certain
      upconversions (or avoid them).

   valuation(sub { ... })
   val(sub { ... })
      Takes a subref, creates a valuation that represents it. The subref will
      get the record as its first and only argument.

      Example(s):
         To get the square of the "x" field:
            val(sub{ $[0]->{x} ** 2 })

   xform(<aggregator>, <snippet>)
      Takes an aggregator and a snippet and produces an aggregator the
      represents invoking the snippet on the aggregator's result.

      Example(s):
         To take the difference between the first and second time fields of the record collection:
            xform(recs(), <<{{1/time}} - {{0/time}}>>)

See Also

RecordStream(3) - Overview of the scripts and the system

recs-examples(3) - A set of simple recs examples

recs-story(3) - A humorous introduction to RecordStream

SCRIPT --help - every script has a --help option, like the output above