NAME
recs-toptable
recs-toptable --help-all
Help from: --help-basic:
Usage: recs-toptable <args> [<files>]
Creates a multi-dimensional pivot table with any number of x and y axises. There is additional help available through --full that includes examples
The x and y rows and columns are the values of the field specified
X and Y fields can take the special value 'FIELD' which uses unused field names as values for the FIELD dimension
--x-field|x Add a x field, values of the specified field will become columns in the table, may be a keyspec or a keygroup
--y-field|y Add a y field, values of the specified field will become rows in the table, may be a keyspec or a keygroup
--v-field|v Specify the value to display in the table, if multiple value fields are specified and FIELD is not placed in the x or y axes, then the last one wins, may be a keyspec or a keygroup. If FIELD is in an
axis, then --v specifies the fields to be included in that expansion
--pin Pin a field to a certain value, only display records matching that value, very similar to doing a recs-grep befor toptable. Takes value of the form: field=pinnedValue, field may be a keyspec (not
a keygroup)
--sort Take sort specifications to sort X values and Y values in headers. See `recs-sort --help` for details of sort specifications, especially the * option to sort "ALL" to the end, e.g. "some_field=lex*".
--noheaders Do not print row and column headers (removes blank rows and columns)
--records|recs Instead of printing table, output records, one per row of the table.
--filename-key|fk <keyspec> Add a key with the source filename (if no filename is applicable will put NONE)
Help Options:
--help-all Output all help for this script
--help This help screen
--help-full Tutorial on toptable, with many examples
--help-keygroups Help on keygroups, a way of specifying multiple keys
--help-keys Help on keygroups and keyspecs
--help-keyspecs Help on keyspecs, a way to index deeply and with regexes
Simple Examples (see --full for more detailed descriptions)
# Collate and display in a nice table
... | recs-collate --key state,priority -a count | recs-toptable --x state --y priority
# Display left over field names as columns
... | recs-collate --key state,priority -a count -a sum,rss | recs-toptable --x state,FIELD --y priority
# Specify the displayed cell values
... | recs-collate --key state,priority -a count -a sum,rss | recs-toptable --x state,FIELD --y priority --value sum_rss
Help from: --help-full:
Full Help
Lets first take a look at some examples:
Lets take this stream, which is a portion of my recs-fromps:
$ recs-fromps --fields rss,pid,state,priority | recs-topn --key state -n 5 | tee /var/tmp/psrecs
{"priority":0,"pid":1,"rss":471040,"state":"sleep"}
{"priority":0,"pid":2,"rss":0,"state":"sleep"}
{"priority":0,"pid":3,"rss":0,"state":"sleep"}
{"priority":0,"pid":4,"rss":0,"state":"sleep"}
{"priority":19,"pid":5,"rss":0,"state":"sleep"}
{"priority":19,"pid":2094,"rss":8351744,"state":"run"}
{"priority":0,"pid":28129,"rss":4784128,"state":"run"}
{"priority":19,"pid":28171,"rss":405504,"state":"run"}
Ok, Now lets get a table out of this, first we'll collate into some useful information:
$ cat /var/tmp/psrecs | recs-collate --perfect --key priority,state -a count
{"priority":0,"count":4,"state":"sleep"}
{"priority":19,"count":1,"state":"sleep"}
{"priority":0,"count":1,"state":"run"}
{"priority":19,"count":2,"state":"run"}
And lets get a table:
$ cat /var/tmp/psrecs | recs-collate --perfect --key priority,state -a count | recs-toptable --x priority --y state
+-----+--------+-+--+
| |priority|0|19|
+-----+--------+-+--+
|state| | | |
+-----+--------+-+--+
|run | |1|2 |
+-----+--------+-+--+
|sleep| |4|1 |
+-----+--------+-+--+
So, you can see that the VALUES of priority and state are used as the columns / rows. So that there is 1 process in state 'run' and priority 0, and 4 in state 'sleep' and priority 0
The --cube option on recs-collate also interacts very well with toptable:
$ cat /var/tmp/psrecs | recs-collate --perfect --key priority,state -a count --cube | recs-toptable --x priority --y state
+-----+--------+-+--+---+
| |priority|0|19|ALL|
+-----+--------+-+--+---+
|state| | | | |
+-----+--------+-+--+---+
|ALL | |5|3 |8 |
+-----+--------+-+--+---+
|run | |1|2 |3 |
+-----+--------+-+--+---+
|sleep| |4|1 |5 |
+-----+--------+-+--+---+
We added an ALL row and an ALL column. So from this you can see that there are 5 processes in priority 0, 3 processes in state 'run' and 8 processes all told in the table (the ALL, ALL intersection)
Now lets see what happens when we have more than 1 left over field. Lets also sum up the rss usage of the processes with -a sum,rss on recs-collate:
$ cat /var/tmp/psrecs | recs-collate --perfect --key priority,state -a count --cube -a sum,rss
{"priority":0,"count":4,"state":"sleep","sum_rss":471040}
{"priority":"ALL","count":5,"state":"sleep","sum_rss":471040}
{"priority":19,"count":1,"state":"sleep","sum_rss":0}
{"priority":0,"count":5,"state":"ALL","sum_rss":5255168}
{"priority":0,"count":1,"state":"run","sum_rss":4784128}
{"priority":"ALL","count":8,"state":"ALL","sum_rss":14012416}
{"priority":"ALL","count":3,"state":"run","sum_rss":13541376}
{"priority":19,"count":3,"state":"ALL","sum_rss":8757248}
{"priority":19,"count":2,"state":"run","sum_rss":8757248}
So now we have 2 left over fields that aren't columns, count and sum_rss. What happens to our table now:
$ cat /var/tmp/psrecs | recs-collate --perfect --key priority,state -a count --cube -a sum,rss | recs-toptable --x priority --y state
+-----+--------+-------+-------+--------+
| |priority|0 |19 |ALL |
+-----+--------+-------+-------+--------+
|state| | | | |
+-----+--------+-------+-------+--------+
|ALL | |5255168|8757248|14012416|
+-----+--------+-------+-------+--------+
|run | |4784128|8757248|13541376|
+-----+--------+-------+-------+--------+
|sleep| |471040 |0 |471040 |
+-----+--------+-------+-------+--------+
We now have sum_rss values in this field. What if we want the other field (count) displayed? We just use --v-field to specify what value field to use:
$ cat /var/tmp/psrecs | recs-collate --perfect --key priority,state -a count --cube -a sum,rss | recs-toptable --x priority --y state --v count
+-----+--------+-+--+---+
| |priority|0|19|ALL|
+-----+--------+-+--+---+
|state| | | | |
+-----+--------+-+--+---+
|ALL | |5|3 |8 |
+-----+--------+-+--+---+
|run | |1|2 |3 |
+-----+--------+-+--+---+
|sleep| |4|1 |5 |
+-----+--------+-+--+---+
Ok, but what if we want to see both left over fields at the same time? What we really want is to add a column or row for each of count and sum_rss. (where the title of the row is count or sum_rss, not the values of the field). We can do
this by using the special FIELD specifier like so:
$ cat /var/tmp/psrecs | recs-collate --perfect --key priority,state -a count --cube -a sum,rss | recs-toptable --x priority,FIELD --y state
+-----+--------+-----+-------+-----+-------+-----+--------+
| |priority|0 | |19 | |ALL | |
+-----+--------+-----+-------+-----+-------+-----+--------+
| |FIELD |count|sum_rss|count|sum_rss|count|sum_rss |
+-----+--------+-----+-------+-----+-------+-----+--------+
|state| | | | | | | |
+-----+--------+-----+-------+-----+-------+-----+--------+
|ALL | |5 |5255168|3 |8757248|8 |14012416|
+-----+--------+-----+-------+-----+-------+-----+--------+
|run | |1 |4784128|2 |8757248|3 |13541376|
+-----+--------+-----+-------+-----+-------+-----+--------+
|sleep| |4 |471040 |1 |0 |5 |471040 |
+-----+--------+-----+-------+-----+-------+-----+--------+
So, now in one table we can see all the intersections of state and priority values with the count and sum_rss fields. Remember that the ALL field (row and column) are provided by the --cube functionality of recs-collate
Now, say you want to pin value, lets just look at processes in state run for instance:
$ cat /var/tmp/psrecs | recs-collate --perfect --cube --key priority,state -a count -a sum,rss | recs-toptable --x priority,FIELD --y state -v sum_rss,count --pin state=run
+-----+--------+-----+-------+-----+-------+-----+--------+
| |priority|0 | |19 | |ALL | |
+-----+--------+-----+-------+-----+-------+-----+--------+
| |FIELD |count|sum_rss|count|sum_rss|count|sum_rss |
+-----+--------+-----+-------+-----+-------+-----+--------+
|state| | | | | | | |
+-----+--------+-----+-------+-----+-------+-----+--------+
|run | |1 |4784128|2 |8757248|3 |13541376|
+-----+--------+-----+-------+-----+-------+-----+--------+
As you can see, this is basically short hand for doing a recs-grep, the transformation to recs group would look like:
$ cat /var/tmp/psrecs | recs-collate --perfect --cube --key priority,state -a count -a sum,rss | recs-grep '$r->{state} eq "run"' | recs-toptable --x priority,FIELD --y state -v sum_rss,count
(which produces the same table as above).
Help from: --help-keygroups:
KEY GROUPS
SYNTAX: !regex!opt1!opt2... Key groups are a way of specifying multiple fields to a recs command with a single argument or function. They are generally regexes, and have several options to control what fields they match. By default
you give a regex, and it will be matched against all first level keys of a record to come up with the record list. For instance, in a record like this:
{ 'zip': 1, 'zap': 2, 'foo': { 'bar': 3 } }
Key group: !z! would get the keys 'zip' and 'zap'
You can have a literal '!' in your regex, just escape it with a \.
Normally, key groups will only match keys whose values are scalars. This can be changed with the 'returnrefs' or rr flag.
With the above record !f! would match no fields, but !f!rr would match foo (which has a value of a hash ref)
Options on KeyGroups:
returnrefs, rr - Return keys that have reference values (default:off)
full, f - Regex should match against full keys (recurse fully)
depth=NUM,d=NUM - Only match keys at NUM depth (regex will match against
full keyspec)
sort, s - sort keyspecs lexically
Help from: --help-keyspecs:
KEY SPECS
A key spec is short way of specifying a field with prefixes or regular expressions, it may also be nested into hashes and arrays. Use a '/' to nest into a hash and a '#NUM' to index into an array (i.e. #2)
An example is in order, take a record like this:
{"biz":["a","b","c"],"foo":{"bar 1":1},"zap":"blah1"}
{"biz":["a","b","c"],"foo":{"bar 1":2},"zap":"blah2"}
{"biz":["a","b","c"],"foo":{"bar 1":3},"zap":"blah3"}
In this case a key spec of 'foo/bar 1' would have the values 1,2, and 3 in the respective records.
Similarly, 'biz/#0' would have the value of 'a' for all 3 records
You can also prefix key specs with '@' to engage the fuzzy matching logic
Fuzzy matching works like this in order, first key to match wins
1. Exact match ( eq )
2. Prefix match ( m/^/ )
3. Match anywehre in the key (m//)
So, in the above example '@b/#2', the 'b' portion would expand to 'biz' and 2 would be the index into the array, so all records would have the value of 'c'
Simiarly, @f/b would have values 1, 2, and 3
You can escape / with a \. For example, if you have a record:
{"foo/bar":2}
You can address that key with foo\/barSee Also
- RecordStream(3) - Overview of the scripts and the system
- recs-examples(3) - A set of simple recs examples
- recs-story(3) - A humorous introduction to RecordStream
- SCRIPT --help - every script has a --help option, like the output above