NAME

Bencher::Scenario::CloneModules - Benchmark various data cloning modules

VERSION

This document describes version 0.051 of Bencher::Scenario::CloneModules (from Perl distribution Bencher-Scenarios-CloneModules), released on 2021-07-31.

SYNOPSIS

To run benchmark with default option:

% bencher -m CloneModules

To run module startup overhead benchmark:

% bencher --module-startup -m CloneModules

For more options (dump scenario, list/include/exclude/add participants, list/include/exclude/add datasets, etc), see bencher or run bencher --help.

DESCRIPTION

Packaging a benchmark script as a Bencher scenario makes it convenient to include/exclude/add participants/datasets (either via CLI or Perl code), send the result to a central repository, among others . See Bencher and bencher (CLI) for more details.

BENCHMARKED MODULES

Version numbers shown below are the versions used when running the sample benchmark.

Clone 0.45

Clone::PP 1.08

Data::Clone 0.004

Sereal::Dclone 0.003

Storable 3.23

BENCHMARK PARTICIPANTS

  • Clone::clone (perl_code)

    Function call template:

    Clone::clone(<data>)
  • Clone::PP::clone (perl_code)

    Function call template:

    Clone::PP::clone(<data>)
  • Data::Clone::clone (perl_code)

    Function call template:

    Data::Clone::clone(<data>)
  • Sereal::Dclone::dclone (perl_code)

    Function call template:

    Sereal::Dclone::dclone(<data>)
  • Storable::dclone (perl_code)

    Function call template:

    Storable::dclone(<data>)

BENCHMARK DATASETS

  • array0

  • array1

  • array10

  • array100

  • array1k

  • array10k

  • hash1k

  • hash10k

BENCHMARK SAMPLE RESULTS

Sample benchmark #1

Run on: perl: v5.34.0, CPU: Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz (4 cores), OS: GNU/Linux LinuxMint version 19, OS kernel: Linux version 5.3.0-68-generic.

Benchmark command (cloning a 10k-element array):

% bencher -m CloneModules --include-datasets array10k

Result formatted as table:

#table1#
+------------------------+-----------+-----------+-----------------------+-----------------------+---------+---------+
| participant            | rate (/s) | time (ms) | pct_faster_vs_slowest | pct_slower_vs_fastest |  errors | samples |
+------------------------+-----------+-----------+-----------------------+-----------------------+---------+---------+
| Clone::clone           |       492 |     2.03  |                 0.00% |               519.53% | 9.1e-07 |      20 |
| Clone::PP::clone       |      1240 |     0.804 |               153.02% |               144.85% | 4.3e-07 |      20 |
| Storable::dclone       |      1380 |     0.727 |               179.88% |               121.36% | 6.4e-07 |      20 |
| Sereal::Dclone::dclone |      2820 |     0.355 |               472.99% |                 8.12% | 2.1e-07 |      21 |
| Data::Clone::clone     |      3050 |     0.328 |               519.53% |                 0.00% | 2.1e-07 |      20 |
+------------------------+-----------+-----------+-----------------------+-----------------------+---------+---------+

The above result formatted in Benchmark.pm style:

         Rate   C:c  CP:c   S:d  SD:d  DC:c 
 C:c    492/s    --  -60%  -64%  -82%  -83% 
 CP:c  1240/s  152%    --   -9%  -55%  -59% 
 S:d   1380/s  179%   10%    --  -51%  -54% 
 SD:d  2820/s  471%  126%  104%    --   -7% 
 DC:c  3050/s  518%  145%  121%    8%    -- 

Legends:
  C:c: participant=Clone::clone
  CP:c: participant=Clone::PP::clone
  DC:c: participant=Data::Clone::clone
  S:d: participant=Storable::dclone
  SD:d: participant=Sereal::Dclone::dclone

The above result presented as chart:

Sample benchmark #2

Benchmark command (cloning a 10k-pair hash):

% bencher -m CloneModules --include-datasets hash10k

Result formatted as table:

#table2#
+------------------------+-----------+-----------+-----------------------+-----------------------+---------+---------+
| participant            | rate (/s) | time (ms) | pct_faster_vs_slowest | pct_slower_vs_fastest |  errors | samples |
+------------------------+-----------+-----------+-----------------------+-----------------------+---------+---------+
| Clone::clone           |       190 |      5.1  |                 0.00% |               117.84% | 7.8e-06 |      20 |
| Clone::PP::clone       |       210 |      4.7  |                 9.40% |                99.12% | 5.5e-06 |      20 |
| Storable::dclone       |       335 |      2.99 |                72.11% |                26.57% | 2.9e-06 |      20 |
| Data::Clone::clone     |       360 |      2.8  |                84.80% |                17.88% | 3.8e-06 |      20 |
| Sereal::Dclone::dclone |       420 |      2.4  |               117.84% |                 0.00% | 2.7e-06 |      20 |
+------------------------+-----------+-----------+-----------------------+-----------------------+---------+---------+

The above result formatted in Benchmark.pm style:

        Rate   C:c  CP:c   S:d  DC:c  SD:d 
 C:c   190/s    --   -7%  -41%  -45%  -52% 
 CP:c  210/s    8%    --  -36%  -40%  -48% 
 S:d   335/s   70%   57%    --   -6%  -19% 
 DC:c  360/s   82%   67%    6%    --  -14% 
 SD:d  420/s  112%   95%   24%   16%    -- 

Legends:
  C:c: participant=Clone::clone
  CP:c: participant=Clone::PP::clone
  DC:c: participant=Data::Clone::clone
  S:d: participant=Storable::dclone
  SD:d: participant=Sereal::Dclone::dclone

The above result presented as chart:

Sample benchmark #3

Benchmark command (benchmarking module startup overhead):

% bencher -m CloneModules --module-startup

Result formatted as table:

#table3#
+---------------------+-----------+-------------------+-----------------------+-----------------------+-----------+---------+
| participant         | time (ms) | mod_overhead_time | pct_faster_vs_slowest | pct_slower_vs_fastest |  errors   | samples |
+---------------------+-----------+-------------------+-----------------------+-----------------------+-----------+---------+
| Storable            |      10   |               6   |                 0.00% |               215.06% |   0.00034 |      21 |
| Sereal::Dclone      |      10   |               6   |                 0.43% |               213.70% |   0.00023 |      20 |
| Clone::PP           |       9   |               5   |                48.96% |               111.50% |   0.00022 |      20 |
| Clone               |       8.6 |               4.6 |                50.27% |               109.67% |   6e-05   |      22 |
| Data::Clone         |       7   |               3   |                76.04% |                78.97% |   0.00014 |      20 |
| perl -e1 (baseline) |       4   |               0   |               215.06% |                 0.00% | 4.6e-05   |      23 |
+---------------------+-----------+-------------------+-----------------------+-----------------------+-----------+---------+

The above result formatted in Benchmark.pm style:

                         Rate     S   S:D   C:P     C   D:C  perl -e1 (baseline) 
 S                    100.0/s    --    0%   -9%  -14%  -30%                 -60% 
 S:D                  100.0/s    0%    --   -9%  -14%  -30%                 -60% 
 C:P                  111.1/s   11%   11%    --   -4%  -22%                 -55% 
 C                    116.3/s   16%   16%    4%    --  -18%                 -53% 
 D:C                  142.9/s   42%   42%   28%   22%    --                 -42% 
 perl -e1 (baseline)  250.0/s  150%  150%  125%  114%   75%                   -- 

Legends:
  C: mod_overhead_time=4.6 participant=Clone
  C:P: mod_overhead_time=5 participant=Clone::PP
  D:C: mod_overhead_time=3 participant=Data::Clone
  S: mod_overhead_time=6 participant=Storable
  S:D: mod_overhead_time=6 participant=Sereal::Dclone
  perl -e1 (baseline): mod_overhead_time=0 participant=perl -e1 (baseline)

The above result presented as chart:

To display as an interactive HTML table on a browser, you can add option --format html+datatables.

HOMEPAGE

Please visit the project's homepage at https://metacpan.org/release/Bencher-Scenarios-CloneModules.

SOURCE

Source repository is at https://github.com/perlancar/perl-Bencher-Scenarios-CloneModules.

BUGS

Please report any bugs or feature requests on the bugtracker website https://rt.cpan.org/Public/Dist/Display.html?Name=Bencher-Scenarios-CloneModules

When submitting a bug or request, please include a test-file or a patch to an existing test-file that illustrates the bug or desired feature.

SEE ALSO

AUTHOR

perlancar <perlancar@cpan.org>

COPYRIGHT AND LICENSE

This software is copyright (c) 2021, 2017, 2016 by perlancar@cpan.org.

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.