NAME

Test::Against::Build - Test CPAN modules against specific Perl build

SYNOPSIS

    my $self = Test::Against::Build->new( {
        build_tree      => '/path/to/top/of/build_tree',
        results_tree    => '/path/to/top/of/results_tree',
        verbose => 1,
    } );

    my $gzipped_build_log = $self->run_cpanm( {
        module_file => '/path/to/cpan-river-file.txt',
        title       => 'cpan-river-1000',
        verbose     => 1,
    } );

    $ranalysis_dir = $self->analyze_cpanm_build_logs( { verbose => 1 } );

    $fcdvfile = $self->analyze_json_logs( { verbose => 1, sep_char => '|' } );

DESCRIPTION

Who Should Use This Library?

This library should be used by anyone who wishes to assess the impact of a given build of the Perl 5 core distribution on the installability of libraries found on the Comprehensive Perl Archive Network (CPAN).

The Problem to Be Addressed

The Perl Annual Development Cycle

Perl 5 undergoes an annual development cycle whose components typically include:

• Individual commits to the Perl 5 git repository

  These commits may be identified by commit IDs (SHAs), branches or tags.

• Release tarballs

  • Monthly development release tarballs

    Whose version numbers follow the convention of 5.27.0, 5.27.1, etc., where the middle digits are always odd numbers.

  • Release Candidate (RC) tarballs

    Whose version numbers follow the convention of 5.28.0-RC1, 5.28.0-RC2, 5.28.1-RC1.

  • Production release tarballs

    Whose version numbers follow the convention of 5.28.0 (new release); 5.28.1, 5.28.2, etc. (maintenance releases).

Measuring the Impact of Changes in Core on CPAN Modules

You can configure, build and install a perl executable starting from any of the above components and you can install CPAN modules against any such perl executable. Given a list of specific CPAN modules, you may want to be able to compare the results you get from trying to install that list against different perl executables built from different commits or releases at various points in the development cycle. To make such comparisons, you will need to have data generated and recorded in a consistent format. This library provides methods for that data generation and recording.

High-Level View of What the Module Does

Tree Structure

For any particular attempt to build a perl executable from any of the starting points described above, Test::Against::Build guarantees that there exists on disk two directory trees:

1 The build tree

The build tree is a directory beneath which perl, other executables and libraries will be installed (or already are installed). As such, the structure of this tree will look like this:

    top_of_build_tree/bin/
                      bin/perl
                      bin/perldoc
                      bin/cpan
                      bin/cpanm
    ...
    top_of_build_tree/lib/
                      lib/perl5/
                      lib/perl5/5.29.0/
                      lib/perl5/site_perl/
    ...
    top_of_build_tree/.cpanm/
    top_of_build_tree/.cpanreporter/

Test::Against::Build presumes that you will be using Miyagawa's cpanm utility to install modules from CPAN. The .cpanm and .cpanreporter directories will be the locations where data concerning attempts to install CPAN modules are recorded.

2 The results tree

The results tree is a directory beneath which data parsed from the .cpanm directory is formatted and stored. Its format looks like this:

    top_of_results_tree/analysis/
                        buildlogs/
                        storage/

The names of the top-level directories are arbitrary; the names of their subdirectories are not. The top-level directories may be located anywhere writable on disk and need not share a common parent directory. It is the Test::Against::Build object which will establish a relationship between the two trees.

Installation of perl and cpanm

Test::Against::Build does not provide you with methods to build or install these executables. It presumes that you know how to build perl from source, whether that be from a specific git checkout or from a release tarball. It further presumes that you know how to install cpanm against that perl. It does provide a method to identify the directory you should use as the value of the -Dprefix= option to Configure. It also provides methods to determine that you have installed perl and cpanm in the expected locations. Once that determination has been made, it provides you with methods to run cpanm against a specific list of modules, parse the results into files in JSON format and then summarize those results in a delimiter-separated-values file (such as a pipe-separated-values (.psv) file).

Why, you may ask, does Test::Against::Build not provide methods to install these executables? There are a number of reasons why not.

• perl and cpanm already installed

  You may already have on disk one or more perls built from specific commits or release tarballs and have no need to re-install them.

• Starting from git commit versus starting from a tarball

  You can build perl either way, but there's no need to have code in this package to express both ways.

• Many ways to configure perl

  perl configuration is a matter of taste. The only thing which this package needs to provide you is a value for the -Dprefix= option. It should go without saying that if want to measure the impact on CPAN modules of two different builds of perl, you should call Configure with exactly the same set of options for each.

The examples below will provide guidance.

METHODS

new()

• Purpose

  Test::Against::Build constructor. Guarantees that the build tree and results tree have the expected directory structure. Determines whether perl and cpanm have already been installed or not.

• Arguments

      my $self = Test::Against::Build->new( {
          build_tree      => '/path/to/top/of/build_tree',
          results_tree    => '/path/to/top/of/results_tree',
          verbose => 1,
      } );

• Return Value

  Test::Against::Build object.

• Comment

Accessors

The following accessors return the absolute path to the directories in their names:

• get_build_tree()

• get_bin_dir()

• get_lib_dir()

• get_cpanm_dir()

• get_cpanreporter_dir()

• get_results_tree()

• get_analysis_dir()

• get_buildlogs_dir()

• get_storage_dir()

is_perl_built()

• Purpose

  Determines whether a perl executable has actually been installed in the directory returned by get_bin_dir().

• Arguments

  None.

• Return Value

  1 for yes; 0 for no.

is_cpanm_built()

• Purpose

  Determines whether a cpanm executable has actually been installed in the directory returned by get_bin_dir().

• Arguments

• Return Value

  1 for yes; 0 for no.

run_cpanm()

• Purpose

  Use cpanm to install selected Perl modules against the perl built for testing purposes.

• Arguments

  Two mutually exclusive interfaces:

  • Modules provided in a list

        $gzipped_build_log = $self->run_cpanm( {
            module_list => [ 'DateTime', 'AnyEvent' ],
            title       => 'two-important-libraries',
            verbose     => 1,
        } );

  • Modules listed in a file

        $gzipped_build_log = $self->run_cpanm( {
            module_file => '/path/to/cpan-river-file.txt',
            title       => 'cpan-river-1000',
            verbose     => 1,
        } );

  Each interface takes a hash reference with the following elements:

  • module_list OR module_file

    Mutually exclusive; must use one or the other but not both.

    The value of module_list must be an array reference holding a list of modules for which you wish to track the impact of changes in the Perl 5 core distribution over time. In either case the module names are spelled in Some::Module format -- i.e., double-colons -- rather than in Some-Module format (hyphens).

  • title

    String which will be used to compose the name of project-specific output files. Required.

  • verbose

    Extra information provided on STDOUT. Optional; defaults to being off; provide a Perl-true value to turn it on. Scope is limited to this method.

analyze_cpanm_build_logs()

• Purpose

  Parse the build.log created by running run_cpanm(), creating JSON files which log the results of attempting to install each module in the list or file.

• Arguments

      $ranalysis_dir = $self->analyze_cpanm_build_logs( { verbose => 1 } );

  Hash reference which, at the present time, can only take one element: verbose. Optional.

• Return Value

  String holding absolute path to the directory holding .log.json files for a particular run of run_cpanm().

• Comment

analyze_json_logs()

• Purpose

  Tabulate the grades (PASS, FAIL, etc.) assigned to each CPAN distribution analyzed in analyze_cpanm_build_logs() and write to a separator-delimited file.

• Arguments

      $fcdvfile = $self->analyze_json_logs( { verbose => 1, sep_char => '|' } );

  Hash reference which, at the present time, can only take only two elements:

  • verbose

    Extra information provided on STDOUT. Optional; defaults to being off; provide a Perl-true value to turn it on. Scope is limited to this method.

  • sep_char

    The separator character used to delimit columns in the output file. Optional; two possibilities:

    • |

      Pipe -- in which case the file extension will be .psv (default).

    • ,

      Comma -- in which case the file extension will be .csv.

• Return Value

  String holding absolute path to the separator-delimited file created by this method. This file will be placed in the storage/ directory in the results tree as described above.

cpanm Test::Against::Dev

perl -MCPAN -e shell
install Test::Against::Dev