NAME
recs-toptable
recs-toptable --help-all
Help from: --help-basic:
Usage: recs-toptable <args> [<files>]
Creates a multi-dimensional pivot table with any number of x and y axises.
There is additional help available through --full that includes examples
The x and y rows and columns are the values of the field specified
X and Y fields can take the special value 'FIELD' which uses unused field
names as values for the FIELD dimension
--x-field|x Add a x field, values of the specified field
will become columns in the table, may be a
keyspec or a keygroup
--y-field|y Add a y field, values of the specified field
will become rows in the table, may be a keyspec
or a keygroup
--v-field|v Specify the value to display in the table, if
multiple value fields are specified and FIELD is
not placed in the x or y axes, then the last one
wins, may be a keyspec or a keygroup. If FIELD
is in an axis, then --v specifies the fields to
be included in that expansion
--pin Pin a field to a certain value, only display
records matching that value, very similar to
doing a recs-grep before toptable. Takes value
of the form: field=pinnedValue, field may be a
keyspec (not a keygroup)
--sort Take sort specifications to sort X values and Y
values in headers. See `recs-sort --help` for
details of sort specifications, especially the *
option to sort "ALL" to the end, e.g.
"some_field=lex*".
--noheaders Do not print row and column headers (removes
blank rows and columns)
--records|recs Instead of printing table, output records, one
per row of the table.
--sort-to-end|sa Sort ALL fields to the end, equivalent to --sort
FIELD=* for each --y and --y field
--filename-key|fk <keyspec> Add a key with the source filename (if no
filename is applicable will put NONE)
Help Options:
--help-all Output all help for this script
--help This help screen
--help-full Tutorial on toptable, with many examples
--help-keygroups Help on keygroups, a way of specifying multiple keys
--help-keys Help on keygroups and keyspecs
--help-keyspecs Help on keyspecs, a way to index deeply and with regexes
Simple Examples (see --full for more detailed descriptions)
# Collate and display in a nice table
... | recs-collate --key state,priority -a count | recs-toptable --x state --y priority
# Display left over field names as columns
... | recs-collate --key state,priority -a count -a sum,rss | recs-toptable --x state,FIELD --y priority
# Specify the displayed cell values
... | recs-collate --key state,priority -a count -a sum,rss | recs-toptable --x state,FIELD --y priority --value sum_rss
Help from: --help-full:
Full Help
Lets first take a look at some examples:
Lets take this stream, which is a portion of my recs-fromps:
$ recs-fromps --fields rss,pid,state,priority | recs-topn --key state -n 5 | tee /var/tmp/psrecs
{"priority":0,"pid":1,"rss":471040,"state":"sleep"}
{"priority":0,"pid":2,"rss":0,"state":"sleep"}
{"priority":0,"pid":3,"rss":0,"state":"sleep"}
{"priority":0,"pid":4,"rss":0,"state":"sleep"}
{"priority":19,"pid":5,"rss":0,"state":"sleep"}
{"priority":19,"pid":2094,"rss":8351744,"state":"run"}
{"priority":0,"pid":28129,"rss":4784128,"state":"run"}
{"priority":19,"pid":28171,"rss":405504,"state":"run"}
Ok, Now lets get a table out of this, first we'll collate into some useful
information:
$ cat /var/tmp/psrecs | recs-collate --perfect --key priority,state -a count
{"priority":0,"count":4,"state":"sleep"}
{"priority":19,"count":1,"state":"sleep"}
{"priority":0,"count":1,"state":"run"}
{"priority":19,"count":2,"state":"run"}
And lets get a table:
$ cat /var/tmp/psrecs | recs-collate --perfect --key priority,state -a count | recs-toptable --x priority --y state
+-----+--------+-+--+
| |priority|0|19|
+-----+--------+-+--+
|state| | | |
+-----+--------+-+--+
|run | |1|2 |
+-----+--------+-+--+
|sleep| |4|1 |
+-----+--------+-+--+
So, you can see that the VALUES of priority and state are used as the columns /
rows. So that there is 1 process in state 'run' and priority 0, and 4 in state
'sleep' and priority 0
The --cube option on recs-collate also interacts very well with toptable:
$ cat /var/tmp/psrecs | recs-collate --perfect --key priority,state -a count --cube | recs-toptable --x priority --y state
+-----+--------+-+--+---+
| |priority|0|19|ALL|
+-----+--------+-+--+---+
|state| | | | |
+-----+--------+-+--+---+
|ALL | |5|3 |8 |
+-----+--------+-+--+---+
|run | |1|2 |3 |
+-----+--------+-+--+---+
|sleep| |4|1 |5 |
+-----+--------+-+--+---+
We added an ALL row and an ALL column. So from this you can see that there are 5
processes in priority 0, 3 processes in state 'run' and 8 processes all told in
the table (the ALL, ALL intersection)
Now lets see what happens when we have more than 1 left over field. Lets also
sum up the rss usage of the processes with -a sum,rss on recs-collate:
$ cat /var/tmp/psrecs | recs-collate --perfect --key priority,state -a count --cube -a sum,rss
{"priority":0,"count":4,"state":"sleep","sum_rss":471040}
{"priority":"ALL","count":5,"state":"sleep","sum_rss":471040}
{"priority":19,"count":1,"state":"sleep","sum_rss":0}
{"priority":0,"count":5,"state":"ALL","sum_rss":5255168}
{"priority":0,"count":1,"state":"run","sum_rss":4784128}
{"priority":"ALL","count":8,"state":"ALL","sum_rss":14012416}
{"priority":"ALL","count":3,"state":"run","sum_rss":13541376}
{"priority":19,"count":3,"state":"ALL","sum_rss":8757248}
{"priority":19,"count":2,"state":"run","sum_rss":8757248}
So now we have 2 left over fields that aren't columns, count and sum_rss. What
happens to our table now:
$ cat /var/tmp/psrecs | recs-collate --perfect --key priority,state -a count --cube -a sum,rss | recs-toptable --x priority --y state
+-----+--------+-------+-------+--------+
| |priority|0 |19 |ALL |
+-----+--------+-------+-------+--------+
|state| | | | |
+-----+--------+-------+-------+--------+
|ALL | |5255168|8757248|14012416|
+-----+--------+-------+-------+--------+
|run | |4784128|8757248|13541376|
+-----+--------+-------+-------+--------+
|sleep| |471040 |0 |471040 |
+-----+--------+-------+-------+--------+
We now have sum_rss values in this field. What if we want the other field
(count) displayed? We just use --v-field to specify what value field to use:
$ cat /var/tmp/psrecs | recs-collate --perfect --key priority,state -a count --cube -a sum,rss | recs-toptable --x priority --y state --v count
+-----+--------+-+--+---+
| |priority|0|19|ALL|
+-----+--------+-+--+---+
|state| | | | |
+-----+--------+-+--+---+
|ALL | |5|3 |8 |
+-----+--------+-+--+---+
|run | |1|2 |3 |
+-----+--------+-+--+---+
|sleep| |4|1 |5 |
+-----+--------+-+--+---+
Ok, but what if we want to see both left over fields at the same time? What we
really want is to add a column or row for each of count and sum_rss. (where the
title of the row is count or sum_rss, not the values of the field). We can do
this by using the special FIELD specifier like so:
$ cat /var/tmp/psrecs | recs-collate --perfect --key priority,state -a count --cube -a sum,rss | recs-toptable --x priority,FIELD --y state
+-----+--------+-----+-------+-----+-------+-----+--------+
| |priority|0 | |19 | |ALL | |
+-----+--------+-----+-------+-----+-------+-----+--------+
| |FIELD |count|sum_rss|count|sum_rss|count|sum_rss |
+-----+--------+-----+-------+-----+-------+-----+--------+
|state| | | | | | | |
+-----+--------+-----+-------+-----+-------+-----+--------+
|ALL | |5 |5255168|3 |8757248|8 |14012416|
+-----+--------+-----+-------+-----+-------+-----+--------+
|run | |1 |4784128|2 |8757248|3 |13541376|
+-----+--------+-----+-------+-----+-------+-----+--------+
|sleep| |4 |471040 |1 |0 |5 |471040 |
+-----+--------+-----+-------+-----+-------+-----+--------+
So, now in one table we can see all the intersections of state and priority
values with the count and sum_rss fields. Remember that the ALL field (row and
column) are provided by the --cube functionality of recs-collate
Now, say you want to pin value, lets just look at processes in state run
for instance:
$ cat /var/tmp/psrecs | recs-collate --perfect --cube --key priority,state -a count -a sum,rss | recs-toptable --x priority,FIELD --y state -v sum_rss,count --pin state=run
+-----+--------+-----+-------+-----+-------+-----+--------+
| |priority|0 | |19 | |ALL | |
+-----+--------+-----+-------+-----+-------+-----+--------+
| |FIELD |count|sum_rss|count|sum_rss|count|sum_rss |
+-----+--------+-----+-------+-----+-------+-----+--------+
|state| | | | | | | |
+-----+--------+-----+-------+-----+-------+-----+--------+
|run | |1 |4784128|2 |8757248|3 |13541376|
+-----+--------+-----+-------+-----+-------+-----+--------+
As you can see, this is basically short hand for doing a recs-grep, the
transformation to recs group would look like:
$ cat /var/tmp/psrecs | recs-collate --perfect --cube --key priority,state -a count -a sum,rss | recs-grep '$r->{state} eq "run"' | recs-toptable --x priority,FIELD --y state -v sum_rss,count
(which produces the same table as above).
Help from: --help-keygroups:
KEY GROUPS
SYNTAX: !regex!opt1!opt2... Key groups are a way of specifying multiple
fields to a recs command with a single argument or function. They are
generally regexes, and have several options to control what fields they
match. By default you give a regex, and it will be matched against all first
level keys of a record to come up with the record list. For instance, in a
record like this:
{ 'zip': 1, 'zap': 2, 'foo': { 'bar': 3 } }
Key group: !z! would get the keys 'zip' and 'zap'
You can have a literal '!' in your regex, just escape it with a \.
Normally, key groups will only match keys whose values are scalars. This can
be changed with the 'returnrefs' or rr flag.
With the above record !f! would match no fields, but !f!rr would match foo
(which has a value of a hash ref)
Options on KeyGroups:
returnrefs, rr - Return keys that have reference values (default:off)
full, f - Regex should match against full keys (recurse fully)
depth=NUM,d=NUM - Only match keys at NUM depth (regex will match against
full keyspec)
sort, s - sort keyspecs lexically
Help from: --help-keyspecs:
KEY SPECS
A key spec is short way of specifying a field with prefixes or regular
expressions, it may also be nested into hashes and arrays. Use a '/' to nest
into a hash and a '#NUM' to index into an array (i.e. #2)
An example is in order, take a record like this:
{"biz":["a","b","c"],"foo":{"bar 1":1},"zap":"blah1"}
{"biz":["a","b","c"],"foo":{"bar 1":2},"zap":"blah2"}
{"biz":["a","b","c"],"foo":{"bar 1":3},"zap":"blah3"}
In this case a key spec of 'foo/bar 1' would have the values 1,2, and 3 in
the respective records.
Similarly, 'biz/#0' would have the value of 'a' for all 3 records
You can also prefix key specs with '@' to engage the fuzzy matching logic
Fuzzy matching works like this in order, first key to match wins
1. Exact match ( eq )
2. Prefix match ( m/^/ )
3. Match anywehre in the key (m//)
So, in the above example '@b/#2', the 'b' portion would expand to 'biz' and 2
would be the index into the array, so all records would have the value of 'c'
Simiarly, @f/b would have values 1, 2, and 3
You can escape / with a \. For example, if you have a record:
{"foo/bar":2}
You can address that key with foo\/barSEE ALSO
See App::RecordStream for an overview of the scripts and the system
Run
recs examplesor see App::RecordStream::Manual::Examples for a set of simple recs examplesRun
recs storyor see App::RecordStream::Manual::Story for a humorous introduction to RecordStreamEvery command has a
--helpmode available to print out usage and examples for the particular command, just like the output above.