/ 
Zonemaster-Engine-v4.5.0
River stage one • 2 direct dependents • 2 total dependents
/ Zonemaster::Engine::Overview

NAME

Zonemaster::Engine::Overview - The Zonemaster Test Engine

INTRODUCTION

The Zonemaster system is a quality control tool for DNS zones, produced in cooperation between AFNIC and IIS (the top-level registries for respectively France and Sweden). It is a successor both to AFNIC's tool Zonecheck and IIS's tool DNSCheck, and is intended to be an improvement over both.

The system as a whole consists of the test engine and, as distributed by the project, two different front ends. One is a command-line interface intended for use by experienced technicians, and one is a web interface meant for use by anyone. This document only talks about the test engine.

DESCRIPTION

Brief overview

Conceptually, the test engine consists of a number of test implementation modules surrounded by a support framework. Anyone wanting to use Zonemaster to perform tests communicates with the framework from the "outside", and all modules implementing tests see the world entirely through the framework. Doing things this way lets us have features like the ability to test domains before they are published, to record entire test runs for later analysis and to make sure that test restults are (as far as reality allows) predictable and repeatable.

For users of Zonemaster

If all you want to do is run tests, you need to care about four or five modules. Zonemaster::Engine is the main access point to the framework, and it is via its methods that you set the configuration (if needed), request that tests be started and access the logger. The logger is where the test results end up, so that's pretty important. On top of those, you may want to use the Zonemaster::Engine::Translator to turn the results into human-readable messages.

There are two ways that you can get the results of a test you've requested: the simple-but-inflexible way and the flexible-but-complex way.

The simple-but-inflexible way is that all the methods in Zonemaster::Engine that run tests return lists of Zonemaster::Engine::Logger::Entry objects. Those lists incude all the results that the writer of the test module(s) considered important enough to return by default. The advantage of this method is that it is extremely easy to use. The following is a functional (if not very useful) way to run a full test and print the results from a command line prompt:

perl -MZonemaster::Engine -E 'say "$_" for Zonemaster::Engine->new->test_zone("example.org")'

The main drawbacks of this method are that there is no choice about what messages to see, and it's entirely synchronous. The code that started the test does not get a chance to do anything at all until the whole test has finished, which may be several minutes later.

To get around those drawbacks there is the flexible-but-complex way, which consists of installing a callback that gets executed every time a message is logged. It's not that much more complicated, code-wise. The following example does roughly the same thing as the one above:

perl -MZonemaster::Engine -E 'Zonemaster::Engine->logger->callback(sub {say "$_[0]"}); Zonemaster::Engine->new->test_zone("example.org");'

If you try running those, you'll notice two main differences. First, the second variant prints results as they are generated. Second, it generates a lot more output. On my machine right now, the first example gives me 94 lines of output. The second example gives me 17684.

You can do pretty much whatever you want with the message objects in the callback (including modifying them, although we don't promise that behavior will stay around). If the callback code throws an exception, and that exception is not a subcless of Zonemaster::Engine::Exception, the callback will be removed. Note also that while the callback is running, the test engine itself is not. So think twice before you do potentially time-consuming tasks (like sticking the message in a database) in the callback. After waiting for responses from remote name servers (which usually stands for more than 90% of the time used), the result logging is the single most time-consuming task in a Zonemaster test run.

From here, you probably want to look at the documentation for Zonemaster::Engine, Zonemaster::Engine::Logger, Zonemaster::Engine::Logger::Entry, Zonemaster::Engine::Config and Zonemaster::Engine::Translator.

For developers of Zonemaster Test Modules

If you want to develop a test module of your own, the standard set of modules serve as examples.

As an entry point to the "inside" of the Zonemaster framework, you want to read Zonemaster::Engine::Zone and follow references from there. Of particular interest after the zone class should be the Zonemaster::Engine::Nameserver and possibly Zonemaster::Engine::Recursor classes.

If you do write your own test module, I would very much appreciate feedback on which parts were tricky to figure out, because I'm honestly not sure what I need to explain here. So please, if there's somethat that you think really needs to be written about, add an issue about at https://github.com/zonemaster/zonemaster-engine/issues.

For developers of the Zonemaster Test Framework

Random recommendations and advice. May be replaced with more coherent developer documentation in the future.

  • Stability, predictability and reliability are more important than performance.

  • Don't forget that starting with Perl version 5.18, the order in which you get the keys out of a hash will be different every time the script is run. Get used to always writing sort keys %hash.

  • If two (or more) test implementation modules implement the same (or very similar) thing, it should probably be extracted into the framework.

  • The unit tests run against pre-recorded data, unless the environment variable ZONEMASTER_RECORD is set to a (perl-)true value. In that case, it runs against the live DNS world and records all results for future use. Unfortunately this sometime means that some tests fail, when we were relying on seeing certain problems in certain domains, and those no longer look the same.

  • The translation strings returned from a test module are used as keys in the GNU gettext system, so if you change anything in them don't forget to also change the translation .po files in share.

  • Adding a new message tag is more work than it first looks, since it needs to be added to the test module metadata, the default policy and the translation system in order to be fully functional.

REFERENCES

https://github.com/zonemaster/zonemaster

Main repository, holding among other things our test specifications.

List of all RFCs referred to in the test specifications