/ 
Template-Benchmark-1.09
River stage one • 1 direct dependent • 1 total dependent
/ benchmark_template_engines

NAME

benchmark_template_engines - Test the relative performance of several different types of template engines.

SYNOPSIS

benchmark_template_engines [options]

Options:
  --help            brief help message
  --man             full documentation
  --duration=N,-d N duration (in seconds) to run each benchmark (default 30)
  --repeats=N,-r N  number of repeated sections in template (default 30)
  --dataset=S       which presupplied dataset to use (default 'original')
  --progress, --noprogress
                    show or hide a progress bar (default hide)
  --json, --nojson  enable or disable JSON-formatted output (default off)
  --quiet, --noquiet
                    show or hide extra output beyond the report (default show)
  --keeptmpdirs, --nokeeptmpdirs
                    keep or remove template/cache dirs (default remove)
  --skipoutputcompare, --noskipoutputcompare
                    skip the check that template outputs matched
                    (default don't skip)
  --style=S         style to pass to Benchmark.pm (default 'none')
  --<feature>, --no<feature>
                    enable or disable a Template::Benchmark feature
  --<feature>=N     enable a feature with a specific number of repeats
  --<cache_type>, --no<cache_type>
                    enable or disable a Template::Benchmark cache type
  --featuresfrom=plugin
                    enable all features supported by that plugin
  --typesfrom=plugin
                    enable all cache types supported by that plugin
  --all             enable all features and cache types
  --allfeatures     enable all teamplate features
  --nofeatures      disable all template features
  --alltypes        enable all cache types
  --notypes         disable all cache types
  --featurematrix   display matrix of what engines support what features
  -I <directory>    push <directory> onto perl's module search path

EXAMPLE OUTPUT

benchmark_template_engines --progress --notypes --uncached_string \
  --onlyplugin TemplateSandbox,HTMLTemplate,TemplateToolkit \
  --onlyplugin TemplateAlloyTT,TemplateAlloyHT
--- Starting Benchmarks --------------------------------------------------------
ETA: 7 benchmarks to run = 70 seconds minimum.
progress: 100% [=====================================================]D 0h01m25s
--- Template Benchmark @Thu Feb 18 20:14:08 2010 -------------------------------
HT         - HTML::Template (2.9)
TAHT       - Template::Alloy (1.013) in HTML::Template mode
TATT       - Template::Alloy (1.013) in Template::Toolkit mode
TS         - Template::Sandbox (1.02_01) without caching
TT         - Template::Toolkit (2.22)
TT_X       - Template::Toolkit (2.22) with Stash::XS (no version number)
TT_XCET    - Template::Toolkit (2.22) with Stash::XS (no version number) and
             Template::Parser::CET (0.05)
--- uncached_string ------------------------------------------------------------
          Rate      TT    TT_X    TATT TT_XCET      TS    TAHT      HT
TT      4.69/s      --    -16%    -56%    -56%    -66%    -73%    -80%
TT_X    5.60/s     19%      --    -47%    -48%    -59%    -68%    -76%
TATT    10.6/s    125%     88%      --     -2%    -23%    -39%    -54%
TT_XCET 10.7/s    129%     92%      2%      --    -21%    -38%    -54%
TS      13.6/s    191%    144%     29%     27%      --    -21%    -41%
TAHT    17.2/s    268%    208%     63%     61%     26%      --    -26%
HT      23.1/s    394%    314%    119%    116%     70%     34%      --

DESCRIPTION

benchmark_template_engines builds a test template according to various commandline parameters (or some "sensible" defaults) and then benchmarks the performance of a number of templating systems at running the template.

It groups the template systems into three main categories for the purposes of comparing like-with-like: those tests that are from a string template held in memory and parsed/compiled on each execution; those that read the template or a compiled version from file on every execution; and those that read the template from file first and then hold the template or a compiled version in memory.

These are roughly analogous to running in a plain CGI environment without caching; running in CGI using file caching; and running under mod_perl with memory caching.

These results may bear no resemblence to real-world performance in absolute terms, however they should provide an indication of the strengths and weaknesses of the different template systems relative to each other.

For more details on how the benchmarks are performed and on various options, consult the documentation for Template::Benchmark.

OPTIONS

--duration=N, -d N

Run the benchmark of each template engine for N seconds. Increasing this will reduce the statistical error on the benchmark results. It will also take the benchmark stript longer to run of course.

If the duration is set to 0 on the commandline then no benchmarks will be timed, however the initial run to test template output will still happen, which may be useful to developers writing new plugins. (Default: 10 seconds.)

--repeats=N, -r N

Repeat the central section of the template N times to simulate a longer template. Tweak this if you want to see behaviour over particularly long or short templates. (Default: 30 repeats.)

--dataset=S

Provide the name of one of the presupplied datasets to use instead of the default. (Default: 'original'.)

--progress, --noprogress

If enabled, this will display a progress bar using Term::ProgressBar::Simple during the benchmark run. The progress bar is only updated between benchmark runs, so while it won't effect timings it will only update every 10 seconds (or whatever you've passed to --duration).

If benchmark_template_engines is run non-interactively or if Term::ProgressBar::Simple is not installed, no progress bar will be displayed. (Default: disabled.)

--quiet, --noquiet

If enabled this option will suppress all output except that of the benchmark report. This includes suppressing any reasons for template engines being skipped. (Default: disabled, ie: show extra stuff.)

--json, --nojson

If enabled this option will output the report as a data-structure encoded using JSON::Any. This could be suitable for storage so that you can record historical benchmark data and read it with a program later.

If you set --json then --quiet will automatically be set also, to prevent extra output messing up the JSON output. (Default: disabled.)

--keeptmpdirs, --nokeeptmpdirs

If enabled, this option will skip the removal of the temporary dirs created to store the generated templates and caches, and on program exit it will print the location of these temporary dirs.

This is useful if you wish to inspect the dirs for debugging purposes.

Note that the output of the dir locations does NOT respect the --quiet option, so if --keeptmpdirs is used in conjunction with --json the output of the directory locations will most likely corrupt the format of your JSON output. (Default: disabled, ie: remove dirs.)

--skipoutputcompare, --noskipoutputcompare

If enabled, this option will skip the sanity check that compares the output of the different template engines to ensure that they're producing the same output.

This is useful as a workaround if the sanity check is producing a mismatch error that you deem to be unimportant. It is strongly recommended that you never use this option without first manually checking the mismatched outputs to be certain that they are in fact unimportant.

(Default: disabled, ie: perform the check.)

--style=S

Passes the argument S as a style to Benchmark, this determines if Benchmark generates any output of its own. See Benchmark for valid settings.

This option may be useful if you want to see the raw timings as Benchmark produces them.

If you set the style to anything other than 'none', it will ignore the --quiet option, and will corrupt any --json enabled output. (Default: none.)

--<feature>, --no<feature>

Enable or disable the named feature within Template::Benchmark when producing the benchmark template.

See the documentation for Template::Benchmark for an accurate and up-to-date listing. (Default: whatever Template::Benchmark sets as defaults.)

--<cache_type>, --no<cache_type>

Enable or disable the type of caching within Template::Benchmark when producing the benchmarks.

Valid values are: uncached_string, uncached_disk, file_cache, shared_memory_cache, memory_cache and instance_reuse. See the documentation for Template::Benchmark for an accurate and up-to-date listing of values and what they mean. (Default: whatever Template::Benchmark sets as defaults.)

--all

Enable all available cache types and template features except those you have explicitly disabled with their corresponding --no<feature> switch.

--nofeatures, --notypes

Disable all available template features or cache types except those you have explicitly enabled with their corresponding --<feature> or --<type> switch.

--allfeaturse, --alltypes

Enable all available template features or cache types except those you have explicitly disabled with their corresponding --no<feature> or --no<type> switch.

--featuresfrom plugin name
--typesfrom plugin name

Sets the features or cache types to be enabled to be all those supported by the given plugin.

Setting either of these options will override all other options for enabling or disabling either features or cache types.

--featurematrix

Print a chart showing a matrix of all template engines vs the chosen template features with a Y or N indicating support for that feature by that engine.

If you wish to see a matrix of all features, make certain to use the --all or --allfeatures options too.

Once the feature matrix is displayed the program exits and no benchmarks will be run.

--syntaxmatrix

Print a chart showing a matrix of all template engines against some common types of template engine syntax with a Y or N indicating if it could be described in that fashion.

The current syntax types are: embedded-perl, mini-language. See the Template::Benchmark::Engine documentation for more information on what these types represent.

A column indicating whether a template engine is implemented in pure-perl is also displayed.

Once the syntax matrix is displayed the program exits and no benchmarks will be run.

-I directory

This will push directory onto perl's module path with use lib '<directory>', this is handy if you want to use a development copy of a plugin or template engine that isn't installed to the usual perl paths.

--onlyplugin plugin name
--skipplugin plugin name

This will push plugin name onto the whitelist or blacklist of template engine plugins to run.

--help, -h, -H, -?

Print a brief help message and exits.

--man

Prints the manual page and exits.

ACKNOWLEDGEMENTS

Thanks to Paul Seamons for creating the the bench_various_templaters.pl script distributed with Template::Alloy, which was the ultimate inspiration for this script.

AUTHOR

Sam Graham, <libtemplate-benchmark-perl at illusori.co.uk>

COPYRIGHT & LICENSE

Copyright 2010 Sam Graham.

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.