NAME

Bencher::Scenario::DataSahResolve::Overhead - Benchmark the overhead of resolving schemas

VERSION

This document describes version 0.004 of Bencher::Scenario::DataSahResolve::Overhead (from Perl distribution Bencher-Scenarios-DataSahResolve), released on 2021-08-01.

SYNOPSIS

To run benchmark with default option:

 % bencher -m DataSahResolve::Overhead

For more options (dump scenario, list/include/exclude/add participants, list/include/exclude/add datasets, etc), see bencher or run bencher --help.

DESCRIPTION

Packaging a benchmark script as a Bencher scenario makes it convenient to include/exclude/add participants/datasets (either via CLI or Perl code), send the result to a central repository, among others . See Bencher and bencher (CLI) for more details.

BENCHMARKED MODULES

Version numbers shown below are the versions used when running the sample benchmark.

Data::Sah 0.909

Data::Sah::Normalize 0.051

Data::Sah::Resolve 0.011

BENCHMARK PARTICIPANTS

• resolve_schema (command)

  Command line:

   #TEMPLATE: #perl -MData::Sah::Resolve=resolve_schema -e for (@{ <schemas> }) { resolve_schema($_) }

• normalize_schema (command)

  Command line:

   #TEMPLATE: #perl -MData::Sah::Normalize=normalize_schema -e for (@{ <schemas> }) { normalize_schema($_) }

• gen_validator (command)

  Command line:

   #TEMPLATE: #perl -MData::Sah=gen_validator -e for (@{ <schemas> }) { gen_validator($_, {return_type=>q(str)}) }

BENCHMARK DATASETS

• int

• perl::modname

• 5-schemas

BENCHMARK SAMPLE RESULTS

Sample benchmark #1

Run on: perl: v5.34.0, CPU: Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz (4 cores), OS: GNU/Linux LinuxMint version 19, OS kernel: Linux version 5.3.0-68-generic.

Benchmark command (default options):

 % bencher -m DataSahResolve::Overhead

Result formatted as table:

 #table1#
 +------------------+---------------+-----------+-----------+-----------------------+-----------------------+---------+---------+
 | participant      | dataset       | rate (/s) | time (ms) | pct_faster_vs_slowest | pct_slower_vs_fastest |  errors | samples |
 +------------------+---------------+-----------+-----------+-----------------------+-----------------------+---------+---------+
 | gen_validator    | 5-schemas     |        19 |        53 |                 0.00% |               609.35% | 0.00015 |      20 |
 | gen_validator    | perl::modname |        23 |        44 |                20.69% |               487.72% | 0.00014 |      20 |
 | gen_validator    | int           |        24 |        41 |                27.73% |               455.34% | 0.00023 |      20 |
 | resolve_schema   | 5-schemas     |        60 |        17 |               214.27% |               125.71% | 0.00013 |      20 |
 | resolve_schema   | perl::modname |        70 |        20 |               250.89% |               102.16% | 0.00015 |      20 |
 | resolve_schema   | int           |        71 |        14 |               275.44% |                88.94% | 0.00012 |      20 |
 | normalize_schema | int           |       100 |         8 |               559.28% |                 7.59% | 0.00018 |      20 |
 | normalize_schema | 5-schemas     |       100 |         8 |               577.87% |                 4.64% | 0.00011 |      20 |
 | normalize_schema | perl::modname |       100 |         7 |               609.35% |                 0.00% | 0.00016 |      20 |
 +------------------+---------------+-----------+-----------+-----------------------+-----------------------+---------+---------+

The above result formatted in Benchmark.pm style:

                      Rate  g_v 5-schemas  g_v perl::modname  g_v int  r_s perl::modname  r_s 5-schemas  r_s int  n_s int  n_s 5-schemas  n_s perl::modname 
  g_v 5-schemas       19/s             --               -16%     -22%               -62%           -67%     -73%     -84%           -84%               -86% 
  g_v perl::modname   23/s            20%                 --      -6%               -54%           -61%     -68%     -81%           -81%               -84% 
  g_v int             24/s            29%                 7%       --               -51%           -58%     -65%     -80%           -80%               -82% 
  r_s perl::modname   70/s           165%               120%     104%                 --           -15%     -30%     -60%           -60%               -65% 
  r_s 5-schemas       60/s           211%               158%     141%                17%             --     -17%     -52%           -52%               -58% 
  r_s int             71/s           278%               214%     192%                42%            21%       --     -42%           -42%               -50% 
  n_s int            100/s           562%               450%     412%               150%           112%      75%       --             0%               -12% 
  n_s 5-schemas      100/s           562%               450%     412%               150%           112%      75%       0%             --               -12% 
  n_s perl::modname  100/s           657%               528%     485%               185%           142%     100%      14%            14%                 -- 
 
 Legends:
   g_v 5-schemas: dataset=5-schemas participant=gen_validator
   g_v int: dataset=int participant=gen_validator
   g_v perl::modname: dataset=perl::modname participant=gen_validator
   n_s 5-schemas: dataset=5-schemas participant=normalize_schema
   n_s int: dataset=int participant=normalize_schema
   n_s perl::modname: dataset=perl::modname participant=normalize_schema
   r_s 5-schemas: dataset=5-schemas participant=resolve_schema
   r_s int: dataset=int participant=resolve_schema
   r_s perl::modname: dataset=perl::modname participant=resolve_schema

The above result presented as chart:

To display as an interactive HTML table on a browser, you can add option --format html+datatables.

HOMEPAGE

Please visit the project's homepage at https://metacpan.org/release/Bencher-Scenarios-DataSahResolve.

SOURCE

Source repository is at https://github.com/perlancar/perl-Bencher-Scenarios-DataSahResolve.

BUGS

Please report any bugs or feature requests on the bugtracker website https://rt.cpan.org/Public/Dist/Display.html?Name=Bencher-Scenarios-DataSahResolve

When submitting a bug or request, please include a test-file or a patch to an existing test-file that illustrates the bug or desired feature.

AUTHOR

perlancar <perlancar@cpan.org>

COPYRIGHT AND LICENSE

This software is copyright (c) 2021, 2017, 2016 by perlancar@cpan.org.

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.

cpanm Bencher::Scenarios::DataSahResolve

perl -MCPAN -e shell
install Bencher::Scenarios::DataSahResolve