/ 
Test-A8N-0.06
River stage zero No dependents
/ Test::A8N

NAME

Test::A8N - Storytest Automation Runner

SYNOPSIS

my $a8 = Test::A8N->new({
    filenames => [qw( cases/test1.tc )],
    file_root => 'cases',
});
$a8->run_tests();

DESCRIPTION

Test::A8N was created as a mechanism for writing and running automated storytests in a human-readable and reusable fashion. Its storytest files are easily readable, are natural to write, and are easy for non-technical users to read, while still being easy for developers to automate.

It works by leveraging Test::FITesque to describe test fixtures in a list style, while providing syntatic sugar for test authors. The tests themselves are written in YAML, and while they have a specific structure, it doesn't limit test author's flexibility. And many of the features of YAML, most notably its concept of creating pointers between different parts of a document, means reusing parts of your tests for use elsewhere is trivial.

Testcase Syntax

An A8N testcase file can consist of mulitple YAML documents, separated by three dashes ---. But the structure for each testcase within a file is the same. It consists of a name, a summary, an optional ID, test instructions, and optional preconditions and postconditions.

The instructions, preconditions, and postconditions contain a list of steps that are to be run as part of the test.

---
NAME:    Administrator changes their timezone
ID:      admin_changes_tz
SUMMARY: Administrators need to have a mechanism for changing their
         timezone.
PRECONDITIONS:
    - ensure user exists: admin
    - ensure timezone is: America/Vancouver
INSTRUCTIONS:
    - login:
        username: admin
        password: testpass
    - goto page: Account Settings
    - verify current timezone is: America/Vancouver
    - change timezone to: Australia/Brisbane
    - verify current timezone is: Australia/Brisbane
POSTCONDITIONS:
    - ensure timezone is: America/Vancouver

Despite the actual order specified in a testcase block, each testcase will run its tests in the order of 1) precondition, 2) instructions, 3) postconditions. If you don't specify an ID, then one will be auto-generated for you based on the testcase name.

Goals of this Testcase Format

When a product manager or some other non-technical business user comes up with a set of initial story tests for some feature, most people will naturally think of writing a list. Even describing how to access some feature, people will fall back to describing a list of steps. We therefore set out to capture that as closely as possible in our tests.

As much as possible, we recommend making your fixture calls, and their arguments, read as much as possible as english phrases, without having unnecessary "no-op" filler that will get in the way.

Testcase Fixtures and Suggestions

From the sample testcase described at the beginning of this section, you can see just how natural this method of writing tests is. The tests don't even need to be automated to be able to follow them, as they're readable enough for a user to navigate a website or command-line to be able to follow the specified steps.

However, once you do automate them, there are some subtleties that we've found work quite well in authoring test cases. Namely in there's a difference between performing an action, and testing the result. For instance:

- change timezone to: <some timezone>
- verify timezone is: <some timezone>
- ensure timezone is: <some timezone>

"change" can be used to set some value using the UI you're testing. "verify" can be used to test that some value is set or is present. "ensure" is a subtle one, but it can be used to change a value only if it isn't already present. It's useful to use in preconditions where you don't care to exercise the UI every time you need to add a user, for instance. If you need a user account simply for testing purposes, then your ensure user exists action can simply drop a user account into a database or onto disk without using the UI. This not only speeds up your tests, but makes sure that you only test user creation within your user account tests, not on every test that requires a user to be present.

METHODS

Accessors

fixture_base

Specifies the base fixture classname to use when running testcases. When test files are found within sub-directories of "file_root", the directory names are converted to class names, and appended to this "fixture_base" value.

file_root

Indicates where your testcase files live. This is important because any directory below this point is assumed to be part of the fixture class name.

filenames

The list of test files you wish Test::A8N to run. If none are specified, the test runner will try to find all files under the "file_root" that has an extension in the "allowed_extensions" list.

verbose

Turns on increasing amounts of debugging output. All debug messages are prefixed with a "#" so that it doesn't interfere with TAP output.

Default: 0

allowed_extensions

Specifies what file extensions are valid testcases. In this way you can mix story tests and unit tests within the same directory.

Default: "st", "tc"

testcase_id

Specifies a testcase ID you wish to run. If unset, it will run all testcases in test files.

Object Methods

run_tests

Calls "run_tests" in Test::A8N::File() in all the Test::A8N::File objects returned by "files".

file_paths

Returns a list of paths to all the testcase files that are to be processed. If nothing is specified in "filenames", or it contains directories, then "file_paths" will search in those sub-directories to find any available testcase files.

files

Returns the contents of "file_paths" as instances of the Test::A8N::File class.

SEE ALSO

Test::A8N::File, Test::FITesque

AUTHORS

Michael Nachbaur <mike@nachbaur.com>, Scott McWhirter <konobi@cpan.org>

LICENSE

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself, either Perl version 5.8.7 or, at your option, any later version of Perl 5 you may have available.

COPYRIGHT

Copyright (C) 2008 Sophos, Plc.