/ 
perl-5.25.9
River stage five • 11849 direct dependents • 33721 total dependents
/ bench.pl

NAME

bench.pl - Compare the performance of perl code snippets across multiple perls.

SYNOPSIS

# Basic: run the tests in t/perf/benchmarks against two or
# more perls

bench.pl [options] -- perlA[=labelA] perlB[=labelB] ...

# run the tests against same perlA 2x, with and without extra
# options

bench.pl [options] -- perlA=fast PerlA=slow -Mstrict -Dpsltoc 

# Run bench.pl's own built-in sanity tests

bench.pl --action=selftest

# Run bench on blead, which is then modified and timed again

bench.pl [options] --write=blead.time -- ./perl=blead
# hack hack hack
bench.pl --read=blead.time -- ./perl=hacked

# You can also combine --read with --write
bench.pl --read=blead.time --write=last.time -- ./perl=hacked

DESCRIPTION

By default, bench.pl will run code snippets found in t/perf/benchmarks (or similar) under cachegrind, in order to calculate how many instruction reads, data writes, branches, cache misses, etc. that one execution of the snippet uses. It will run them against two or more perl executables and show how much each test has gotten better or worse.

It is modelled on the perlbench tool, but since it measures instruction reads etc., rather than timings, it is much more precise and reproducible. It is also considerably faster, and is capable of running tests in parallel (with -j). Rather than displaying a single relative percentage per test/perl combination, it displays values for 13 different measurements, such as instruction reads, conditional branch misses etc.

There are options to write the raw data to a file, and to read it back. This means that you can view the same run data in different views with different selection and sort options. You can also use this mechanism to save the results of timing one perl, and then read it back while timing a modification, so that you dont have rerun the same tests on the same perl over and over, or have two perls built at the same time.

The optional =label after each perl executable is used in the display output. If you are doing a two step benchmark then you should provide a label for at least the "base" perl.

OPTIONS

  • --action=foo

    What action to perform. The default is grind, which runs the benchmarks using cachegrind as the back end. The only other action at the moment is selftest, which runs some basic sanity checks and produces TAP output.

  • --average

    Only display the overall average, rather than the results for each individual test.

  • --benchfile=foo

    The path of the file which contains the benchmarks (t/perf/benchmarks by default).

  • --bisect=field,minval,maxval

    Run a single test against one perl and exit with a zero status if the named field is in the specified range; exit 1 otherwise. It will complain if more than one test or perl has been specified. It is intended to be called as part of a bisect run, to determine when something changed. For example,

    bench.pl -j 8 --tests=foo --bisect=Ir,100,105 --perlargs=-Ilib \
    ./miniperl

    might be called from bisect to find when the number of instruction reads for test foo falls outside the range 100..105.

  • --compact=<Iperl>

    Display the results for a single perl executable in a compact form. Which perl to display is specified in the same manner as --norm.

  • --debug

    Enable verbose debugging output.

  • --fields=a,b,c

    Display only the specified fields; for example,

    --fields=Ir,Ir_m,Ir_mm

    If only one field is selected, the output is in more compact form.

  • --grindargs=foo

    Optional command-line arguments to pass to all cachegrind invocations.

    This option is appended to those which bench.pl uses for its own purposes; so it can be used to override them (see --debug output below), and can also be 'abused' to add redirects into the valgrind command invocation.

    For example, this writes PERL_MEM_LOG activity to foobar.$$, because 3>foobar.$$ redirects fd 3, then perl under PERL_MEM_LOG writes to fd 3.

    $ perl Porting/bench.pl --jobs=2 --verbose --debug \
   --tests=call::sub::amp_empty \
   \
   --grindargs='--cachegrind-out-file=junk.$$ 3>foobar.$$' \
   -- \
   perl5.24.0	perl5.24.0:+memlog:PERL_MEM_LOG=3mst

    for the +memlog tests, this executes as: (shown via --debug, then prettyfied)

    Command: PERL_HASH_SEED=0 PERL_MEM_LOG=3mst
  valgrind --tool=cachegrind  --branch-sim=yes
  --cachegrind-out-file=/dev/null --cachegrind-out-file=junk.$$
  3>foobar.$$ perl5.24.0  - 10 2>&1

    The result is that a set of junk.$$ files containing raw cachegrind output are written, and foobar.$$ contains the expected memlog output.

    Notes:

    Theres no obvious utility for those junk.$$ and foobar.$$ files, but you can have them anyway.

    The 3 in PERL_MEM_LOG=3mst is needed because the output would otherwize go to STDERR, and cause parse_cachegrind() to reject the test and die.

    The --grindargs redirect is needed to capture the memlog output; without it, the memlog output is written to fd3, around parse_cachegrind and effectively into /dev/null

    PERL_MEM_LOG is expensive when used.

    call::sub::amp_empty &foo function call with no args or body

         perl5.24.0 perl5.24.0+memlog
     ---------- -----------------
  Ir      394.0          543477.5
  Dr      161.0          146814.1
  Dw       72.0          122304.6
COND       58.0           66796.4
 IND        5.0            5537.7

    COND_m 0.0 6743.1 IND_m 5.0 1490.2

    Ir_m1        0.0             683.7
Dr_m1        0.0              65.9
Dw_m1        0.0               8.5

Ir_mm        0.0              11.6
Dr_mm        0.0              10.6
Dw_mm        0.0               4.7

  • ---help

    Display basic usage information.

  • -j N --jobs=N

    Run N jobs in parallel (default 1). This determines how many cachegrind process will running at a time, and should generally be set to the number of CPUs available.

  • --norm=foo

    Specify which perl column in the output to treat as the 100% norm. It may be a column number (0..N-1) or a perl executable name or label. It defaults to the leftmost column.

  • --perlargs=foo

    Optional command-line arguments to pass to each perl-under-test (perlA, perlB in synopsis) For example, --perlargs=-Ilib.

  • --raw

    Display raw data counts rather than percentages in the outputs. This allows you to see the exact number of intruction reads, branch misses etc. for each test/perl combination. It also causes the AVERAGE display per field to be calculated based on the average of each tests's count rather than average of each percentage. This means that tests with very high counts will dominate.

  • --sort=field:perl

    Order the tests in the output based on the value of field in the column perl. The perl value is as per --norm. For example

    bench.pl --sort=Dw:perl-5.20.0 \
    perl-5.16.0 perl-5.18.0 perl-5.20.0

  • -r file --read=file

    Read in saved data from a previous --write run from the specified file.

    Requires JSON::PP to be available.

  • --tests=FOO

    Specify a subset of tests to run (or in the case of --read, to display). It may be either a comma-separated list of test names, or a regular expression. For example

    --tests=expr::assign::scalar_lex,expr::assign::2list_lex
--tests=/^expr::/

  • --verbose

    Display progress information.

  • -w file --write=file

    Save the raw data to the specified file. It can be read back later with --read. If combined with --read then the output file will be the merge of the file read and any additional perls added on the command line.

    Requires JSON::PP to be available.