/ 
Devel-PatchPerl-2.10-TRIAL
River stage two • 7 direct dependents • 18 total dependents
/ Testing

NAME

Testing - Helper functions for testing Pod-Html

SYNOPSIS

use Testing qw( setup_testing_dir xconvert );

my $tdir = setup_testing_dir( {
    debug       => $debug,
} );

$args = {
    podstub => "htmldir1",
    description => "test --htmldir and --htmlroot 1a",
    expect => $expect_raw,
    p2h => {
        podpath => File::Spec::Unix->catdir($relcwd, 't') . ":" .
                   File::Spec::Unix->catdir($relcwd, 'corpus/test.lib'),
        podroot => catpath($v, '/', ''),
        htmldir => 't',
        quiet   => 1,
    },
    debug => $debug,
};
xconvert($args);

DESCRIPTION

This module exports, upon request only, 2 subroutines which are used in most of the files in the core distribution test suite for Pod-HTML (ext/Pod-Html/t/*.t). In the future we may add additional subroutines, particularly to better diagnose problems with Pod-Html.

Pod-Html's Testing Structure

As of version 1.26 of this module (early 2021), the testing structure consists of 16 .pod files and 18 .t files located in two subdirectories, corpus/ and t/. Let's analyze these by directory.

Files in corpus/

There are currently 2 .pod files in corpus/ both of which are old versions of pod/*.pod files selected to give some complexity to the test suite. Since we don't actually attempt to make HTML out of their POD, we don't need to discuss them further.

Files in t/

There are currently 14 .pod files and 18 .t files in t/. Both of these numbers may change in the future.

Currently there are 2 t/.t files (t/anchorify.t and t/eol.t) which exercise certain functionality of Pod::Html but which do not require t/*.pod files as data input. These files do not make use of the subroutines exported by this module. We may add more test files like this in the future to ensure high test coverage, but don't need to discuss them further here.

The remaining 16 t/*.t test programs make use of the testing subroutines exported by this module. Most, but not all, of these test programs make use of the t/*.pod files. Each such test program makes use of only 1 t/*.pod file at a time, though there are several cases where several, similarly named, test programs make use of the same t/*.pod file for data input. For example,

t/crossref.t
t/crossref2.t
t/crossref3.t

all make use of

t/crossref.pod

Each t/*.pod file consists solely of simple documentation in POD format.

High-level description of programs which use .pod files as input

Each of the t/*.t programs which makes use of a given t/*.pod file slurps the text of a single such t/*.pod file into memory. The test program holds text in a DATA handle which serves as a template for the HTML expected to be generated by running the t/*.pod file through Pod::Html::pod2html(). The HTML output by Pod::Html::pod2html() can vary greatly, particularly with respect to links, depending on the arguments passed to that function. The HTML output will also be affected by the underlying operating system, e.g., with respect to path separators. Hence, we cannot hard-code the expected HTML output into the DATA template or any place else. We have to allow Pod::Html::pod2html() to massage the template data to get an "expected output" against which we match the "actual output" which comes from running Pod::Html::pod2html() over the text originally slurped into memory from the t/*.pod file.

Granted, there is a certain amount of circularity in this testing regimen. On a given operating system, with a given t/*.pod file as raw input, a given POD parser invoked within Pod::Html::pod2html() and a given set of arguments passed to pod2html(), there can and should be only one possible HTML string generated as output. What we currently have in a given test program's DATA handle is merely that HTML string retrofitted with certain template elements as needed to make the "got" and the "expected" identical. We're not testing whether we're generating "good" HTML. We're simply testing that we get consistent results out of pod2html() year after year.

How a test program works step-by-step

Here we continue to focus on those test programs which make use of the testing functions exported by Testing and which take a t/*.pod file as input.

We assume that we begin our tests from the top level of the Perl 5 core distribution and are using t/harness. Hence, to run the test files we say:

cd t; ./perl harness ../ext/Pod-Html/t/*.t; cd -

The program then slurps contents of the DATA handle into memory.

The program then calls setup_testing_dir() from this module to create a temporary directory and populate it as needed. setup_testing_dir() returns the absolute path to that directory, but at the point where that subroutine returns you are actually located two levels beneath the temporary directory in a directory whose relative path is ext/Pod-Html/. (This is equivalent to being in toplevel/ext/Pod-Html/ for tests in versions of Pod-Html distributed with earlier versions of perl.)

Note that this means that at the end of the program you will have to switch back to your starting directory so that the tempdir can automatically be cleaned up. We automate this via an END block.

You then prepare arguments for our principal testing function, xconvert() (which supersedes the former convert_n_test(). These arguments take the form of a single hash reference. One customary but optional element in that hashref, p2h, is itself a hashref of key-value pairs corresponding to switches passed to the pod2html command-line utility or to Pod::Html::pod2html(). The other elements in the hashref passed to xconvert() include the stub of the basename of the t/*.pod file being used, the text of that file (which we've already slurped into memory), the test description, and whether we want extra debugging output or not. The program then adds a key-value pair to indicate whether we're running via core distribution test harness or not.

The hashref is then passed to xconvert() which internally generates an expected HTML output string by massaging the text read in from the DATA handle. xconvert() reads in the relevant t/*.pod file and passes it to Pod::Html::pod2html(), which parses the POD and generates the actual HTML output. If "got" matches "expected", a PASS is recorded for this instance of xconvert().

As the example of t/htmldir1.t illustrates:

  • The user can define a variety of arguments to be passed through to Pod::Html::pod2html().

    my ($v, $d) = splitpath(cwd(), 1);
my @dirs = splitdir($d);
shift @dirs if $dirs[0] eq '';
my $relcwd = join '/', @dirs;

$args = {
    ...
    p2h => {
        podpath => File::Spec::Unix->catdir($relcwd, 't') . ":" .
                   File::Spec::Unix->catdir($relcwd, 'corpus/test.lib'),
        podroot => catpath($v, '/', ''),
        htmldir => 't',
        quiet   => 1,
    },
    ...
};

  • The user can try out a variety of different arguments in the p2h element and end up with the same HTML output as predicted by the DATA template by calling xconvert() more than once per file.

    $args = {
    podstub => "htmldir1",
    description => "test --htmldir and --htmlroot 1a",
    expect => $expect_raw,
    p2h => {
        podpath => File::Spec::Unix->catdir($relcwd, 't') . ":" .
                   File::Spec::Unix->catdir($relcwd, 'corpus/test.lib'),
        podroot => catpath($v, '/', ''),
        htmldir => 't',
        quiet   => 1,
    },
};
xconvert($args);

$args = {
    podstub => "htmldir1",
    description => "test --htmldir and --htmlroot 1b",
    expect => $expect_raw,
    p2h => {
        podpath     => $relcwd,
        podroot     => catpath($v, '/', ''),
        htmldir     => catdir($relcwd, 't'),
        htmlroot    => '/',
        quiet       => 1,
    },
};
xconvert($args);

    Note that in the two "runs" above, the values for podstub are the same, but the arguments to p2h differ; we've distinguished the two runs by different values for description.

Note that all runs within an individual t/*.t program share the same temporary directory. Since Pod::Html::pod2html() typically caches its understanding of where .pod files are located, there is a possibility that the contents of the cache may affect the generated HTML output in an adverse way. This possibility will be addressed in an upcoming version of this program.

When all runs have been completed (as noted above), the END block brings us back to the directory we started from to permit the temporary directory and its contents to be cleanly deleted.

SUBROUTINES

setup_testing_dir()

  • Purpose

    Create and populate a temporary directory to hold all activity for a single t/*.t program.

  • Arguments

    $tdir = setup_testing_dir( {
    startdir    => $startdir,
    debug       => $debug,
} );

    Single hash reference with two possible elements.

    • debug

      A Boolean which you will typically set at the start of your program. A Perl-true value prints out your location and creates a temporary directory which is not cleaned up at the program's completion, thereby permitting you to examine the intermediate files created by the program.

  • Return Value

    String holding the absolute path of the temporary directory.

  • Comments

    The function chdirs internally and leaves you in a directory called ext/Pod-Html beneath the temporary directory found in the return value.

    The function is somewhat equivalent to testing helper function make_test_dir in t/pod2html-lib.pl in versions of Pod-Html shipped with versions of perl up through 5.32.

xconvert()

  • Purpose

    Compare whether the HTML generated by Pod::Html::pod2html()'s parsing of a .pod file matches the expectation generated by parsing the DATA block within the test file.

  • Arguments

    Single hash reference.

    $args = {
    podstub => "htmldir5",
    description => "test --htmldir and --htmlroot 5",
    expect => $expect_raw,
    p2h => {
        podpath     => 't:corpus/test.lib',
        podroot     => $cwd,
        htmldir     => $cwd,
        htmlroot    => '/',
        quiet       => 1,
    },
    debug => $debug,
};
$args->{core} = 1 if $ENV{PERL_CORE};

    Elements are as follows:

    • podstub

      String holding the stub (or stem) of the .pod file being used as input. The stub is the basename of the file less the file extension or suffix. (Equivalent to the first argument passed to the former convert_and_test test helper routine.) Required.

    • description

      String holding the description (or name or label) in typical TAP syntax. (Equivalent to the second argument passed to the former convert_and_test helper routine.) Required.

    • expect

      String holding the "raw" expectations read in from the DATA handle. Each run of xconvert() within a given test file should have the same value for this key. Required.

    • p2h

      Hash reference holding arguments passed to Pod::Html::pod2html() (though without the leading double hyphens (--). See documentation for Pod::Html. Optional, but mostly necessary. In particular, if a .pod file contains any L<> tags, a podpath element almost always needs to be supplied with a colon-delimited list of directories from which to begin a search for files containing POD.

    • debug

      Boolean, generally set once at the program's top. When Perl-true, displays extra debugging output, including turning on Pod::Html::pod2html()'s verbose option. Optional.

    • core

      Boolean. This should be set to a Perl-true value when the file is to be run from the test harness rather than from the top-level of the repository.

  • Return Value

    Not explicitly defined, but should return a Perl-true value upon completion.

  • Comment

    This function essentially asks, "Are we getting the same HTML output the last time we tinkered with the code in this distribution?" Hence, it is dependent on the particular parsing and HTML composition functionality found within Pod::Html::pod2html(), which is a somewhat customized subclass of Pod::Simple::XHTML. If, in the future, we offer functionality based on other parsing classes, then the DATA sections of the t/*.t files will have to be revised and perhaps the guts of xconvert() as well.

    This function is roughly equivalent to test helper function convert_n_test() in earlier versions of Pod-Html.

record_state_of_cache()

  • Purpose

    During debugging, enable developer to examine the state of the Pod-Html cache after each call to xconvert().

  • Arguments

    Single hash reference.

    record_state_of_cache( {
    outdir => "$ENV{P5P_DIR}/pod-html",
    stub => $args->{podstub},
    run => 1,
} );

    Hash reference has the following key-value pairs:

    • outdir

      Any directory of your system to which you want a sorted copy of the cache to be printed.

    • stub

      The same value you passed in $args to xconvert().

    • run

      Integer which you set manually to distinguish among multiple runs of this function within the same test file (presumably corresponding to multiple invocations of xconvert()).

  • Return Value

    Implicitly returns Perl-true value.

  • Comment

    Function will print out location of cache files and other information.

AUTHORS

The testing code reworked into its present form has many authors and dates back to the dawn of Perl 5, perhaps beyond. The documentation was written by James E Keenan in March 2021.