NAME

Bigtop::Parser - the Parse::RecDescent grammar driven parser for bigtop files

SYNOPSIS

Make a file like this:

config {
    base_dir `/home/username`;
    Type1 Backend {}
    Type2 Backend {}
    Type3 Backend {}
}
app App::Name {
    table name { }
    controller SomeController {}
}

Then run this command:

bigtop my.bigtop all

DESCRIPTION

This module is really only designed to be used by bigtop. It provides the grammar which understands bigtop files and turns them into syntax trees. It provides various utility functions for bigtop (or similar tools you might write) and for backends.

Reading further is an indication that you are interested in working on Bigtop and not just in using it to serve your needs.

METHODS

In this section, the methods are grouped, so that similar ones appear together.

METHODS which drive generation for scripts

gen_from_file

The bigtop script calls this method.

You can call this as a class method passing it the name of the bigtop file to read and a list of things to build.

The method is actually quite simple. It merely reads the file, then calls gen_from_string.

gen_from_string

The bigtop script calls this method when --new is used.

This method orchestrates the build. It is called internally by gen_from_file. Call it as a class method. Pass it the bigtop string, the name of the file from which that string came (or undef), and the list of things to build.

The file name is used by Bigtop::Init::Std to copy the bigtop file from its original location into the docs subdirectory of the build directory. If the file name is not defined, it skips that step.

The list of things to build can include any backend type listed in the config block and/or the word 'all'. 'all' will be replaced with a list of all the backend types in the config section (in the order they appear there), as if they had been passed in.

It is legal to mention the same backend more than once. For instance, you could call gen_from_string directly

Bigtop::Parser->gen_from_string(
        $bigtop_string, 'file.bigtop', $create, 'Init', 'Control', 'Init'
);

or equivalently, and more typically, you could call gen_from_file:

Bigtop::Parser->gen_from_file(
    'file.bigtop', $create, 'Init', 'Control', 'Init'
);

Either of these might be useful, if the first Init sets up directories that the Control backend needs, but the generated output from Control should influence the contents of file which Init finally builds. Check your backends for details.

gen_from_string internals

gen_from_string works like this. First, it attempts to parse the config section of the bigtop string. If that works, it iterates through each backend mentioned there building a list of modules to require. This includes looking in backend blocks for template statements. Their values must be template files relative to the directory from which bigtop was invoked.

Once the list is built, it calls its own import method to require them. This allows each backend to register its keywords. If any keyword used in the app section is not registered, a fatal parse error results.

Once the backends are all required, gen_from_string parses the whole bigtop string into an abstract syntax tree (AST). Then it iterates through the build list calling gen_Type on each element's backend. So this:

config {
    Init Std      {}
    SQL  Postgres { template `postgres.tt`; }
}
app ...

Bigtop::Parser->gen_from_string(
        $bigtop_string, 'file.bigtop', 'Init', 'SQL'
);

Results first in the loading of Bigtop::Init::Std and Bigtop::SQL::Postgres, then in calling gen_Init on Init::Std and gen_SQL on SQL::Postgres. During the loading, setup_template is called with postgres.tt on SQL::Postgres.

gen_* methods are called as class methods. They receive the build directory, the AST, and the name of the bigtop_file (which could be undef). Backends can do whatever they like from there. Typically, they put files onto the disk. Those files might be web server conf files, sql to build the database, control modules, templates for viewing, models, etc.

METHODS which invoke the grammar

parse_config_string

Called as a class method (usually by gen_from_string), this method receives the bigtop input as a string. It attempts to parse only the config section which it returns as an AST. Syntax errors in the config section are fatal. Errors in the app section are not noticed.

parse_file

Call this as a class method, passing it the file name to read. It reads the file into memory, then calls parse_string, returning whatever it returns.

parse_string

Call this as a class method, passing it the bigtop string to parse. It calls the grammar to turn the input into an AST, which it returns.

METHODS which control which simple statement keywords are legal

add_valid_keywords

The grammar of a bigtop file is structured, but the legal keywords in its simple statements are defined by the backends (excepts that the config keywords are defined by this module, see Config Keywords below for those).

If you are writing a backend, you should use the base module for your backend type. This will register the standard keywords for that type. For example, suppose you are writing Bigtop::SQL::neWdB. It should be enough to say:

use Bigtop::SQL;

in your module.

If you need to add additional keywords that are specific to your backend, put them in a begin block like this:

BEGIN {
    Bigtop::Parser->add_valid_keywords(
        $type,
        qw( your keywords here),
    );
}

Here $type is the name of the surrounding block in which this keyword will make a valid statement. For example, if $type above is 'app' then this would be legal:

app App::Name {
    your value;
}

The type must be one of these levels:

app

config

controller

field

method

These correspond exactly to the block types in the grammar (except that sequence blocks must currently be empty, in the future sequence will be added to the above list).

is_valid_keyword

Call this as a class method, passing it a type of keyword and a word that might be a valid keyword of that type.

Returns true if the keyword is valid, false otherwise.

get_valid_keywords

Call this as a class method passing it the type of keywords you want.

Returns a list of all registered keywords, of the requested type, in string sorted order.

The two preceding methogs are really for internal use in the grammar.

METHODS which work on the AST

walk_postorder

Walks the AST for you, calling you back when it's time to build something.

The most common skeleton for gen_Backend is:

use Bigtop;
use Bigtop::Backend;

sub gen_Backend {
    my $class     = shift;
    my $build_dir = shift;
    my $tree      = shift;

    # walk the tree
    my $something     = $tree->walk_postoder( 'output_something' );
    my $something_str = join '', @{ $something };

    # write the file
    Bigtop::write_file( $build_dir, $something_string );
}

This walks the tree from the root. The walking is postorder meaning that all children are visited before the current node. Each walk_postorder returns an array reference (which is why we have to join the result in the above skeleton). After the children have been visited, the callback (output_something in the example) is called with their output array reference. You can also pass an additional scalar (which is usually a hash reference) to walk_postorder. It will be passed along to all the child walk_postorders and to the callbacks.

With this module walking the tree, all you must do is provide the appropriate callbacks. Put one at each level of the tree that interests you.

For example, if you are generating SQL, you need to put callbacks in the following packages:

sql_block
sequence_body
table_body
table_element_block
field_body
field_statement

This does require some knowledge of the tree. Please consult the grammar for the possible packages (or grep for package on this file).

The callbacks are called as methods on the current tree node. They receive the output array reference from their children and the data scalar that was passed to walk_postorder (if one was passed in the top level call). So, a typical callback method might look like this:

sub output_something {
    my $self         = shift;
    my $child_output = shift;
    my $data         = shift;
    ...
    return [ $output ];
}

Remember that they must return an array reference. If you need something fancy, you might do this:

return [ [ type1_output => $toutput, type2_output => $other_out ] ];

Then the parent package's callback will receive that and must tease apart the the two types. Note that I have nested arrays here. This prevents two children from overwriting each other's output if you are ever tempted to try saving the return list directly to a hash (think recursion).

(walk_postorder also passes the current node to each child after the data scalar. This is the child's parent, which is really only useful during parent building inside the grammar. The parent comes after the data scalar in both walk_postorder and in the callback. Most backends will just peek in $self->{__PARENT__} which is gauranteed to have the parent once the grammar finishes with the AST.)

set_parent

This method is the callback used by the grammar to make sure that all nodes know who their daddy is. You shouldn't call it, but looking at it shows what the simplest callback might look like. Note that there is only one of these and it lives in the application_ancestor package, which is not one of the packages defined in the grammar. But, this module makes sure that all the grammar defined packages inherit from it.

build_lookup_hash

This method builds the lookup hash you can use to find data about other parts of the tree, without walking to it.

The AST actually has three keys: configuration, application, and lookup. The first two are built in the normal way from the input file. They are genuine ASTs in their own right. The lookup key is not. It does not preserve order. But it does make it easier to retrieve things.

For example, suppose that you are in the method_body package attempting to verify that requested fields for this method are defined in the table for this controller. You could walk the tree, but the lookup hash makes it easier:

unless (
    defined $tree->{lookup}{tables}{$table_name}{fields}{$field_name}
) {
    die "No such column $field_name\n";
}

The easiest way to know what is available is to dump the lookup hash. But the pattern is basically this. At the top level there are fixed keywords for the app level block types: tables, sequences, controllers. The next level is the name of a block. Under that, there is a fixed keyword for each subblock type, etc.

METHODS for use in walk_postorder callbacks

dumpme

Use this method instead of directly calling Data::Dumper::Dump.

While you could dump $self, that's rather messy. The problem is the parent nodes. Their presence means a simple dump will always show the whole app AST. This method carefully removes the parent, dumps the node, and restores the parent, reducing clutter and leaving everything in tact.

get_appname

Call this on the full AST. It returns the name of the application.

get_config

Call this on the full AST. It returns the config subtree.

get_controller_name

Call this, from the method_body package, on the AST node ($self in the callback). Returns the name of the controller for this method. This is useful for error reporting.

get_method_name

Call this, from the method_body package, on the AST node ($self in the callback). Returns the name of this method. Useful for error reporting.

get_name

While this should work everywhere, it doesn't. Some packages have it. If yours does, call it. Otherwise peek in $self->{__NAME__}. But, remember that not everything has a name.

get_table_name

Call this, from the method_body package, on the AST node ($self in the callback). Returns the name of the table this controller controls. Useful for error reporting.

METHODS used internally

import

You probably don't need to call this. But, if you do, pass it a list of backends to import like this:

use Bigtop::Parser qw( Type=Backend=template.tt );

This will load Bigtop::Type::Backend and tell it to use template.tt. You can accomplish the same thing by directly calling import as a class method:

Bigtop::Parser->import( 'Type=Backend=template.tt' );

fatal_error_two_lines

This method is used by the grammar to report fatal parse error in the input. It actually gives 50 characters of trailing context, not two lines, but the name stuck.

fatal_keyword_error

This method is used by the grammer to report on unregistered (often misspelled) keywords. It identifies the offending keyword and the line where it appeared in the input, gives the remainder of the line on which it was seen (which is sometimes only whitespace), and lists the legal choices (often wrapping them in an ugly fashion).

Config KEYWORDS

For simplicity, all config keywords are defined in this module. This is not necessarily ideal and is subject to change.

base_dir

Used only if you supply the --create flag to bigtop (or set create to true when calling gen_from_file or gen_from_string as class methods of this module).

When in create mode, the build directory will be made as a subdirectory of the base_dir. For instance, I could use my home directory:

base_dir `/home/username`;

Note that you need the backquotes to hide the slashes. Also note, that you should use a path which looks good on your development system. In particular, this would work on the appropriate platform:

base_dir `C:\path\to\build`;

The default base_dir is the current directory from which bigtop is run.

app_dir

Used only if you supply the --create flag to bigtop (or set create to true when calling gen_from_file or gen_from_string as class methods of this module).

When in create mode, the actual generated files will be placed into base_dir/app_dir (where the slash is correctly replaced with your OS path separator). If you are in create mode, but don't supply an app_dir, a default is formed from the app name in the manner h2xs would use. Consider:

config {
    base_dir `/home/username`;
}
app App::Name {
}

In this case the app_dir is App-Name. So the build directory is

/home/username/App-Name

By specifying your own app_dir statement, you have complete control of where the app is initially built. For example:

config {
    base_dir `/home/username`;
    app_dir  `myappdir`;
}
app App::Name { }

Will build in /home/username/myappdir.

When not using create mode, all files will be built under the current directory. If that directory doesn't look like an app build directory, a fatal error will result. Either move to the proper directory, or use create mode to avoid the error.

engine

This is passed directly to the use Framework; statement of the top level controller.

Thus,

engine MP13;

becomes something like this:

use Framework qw/ engine=MP13 /;

in the base level controller. Both Catalyst and Gantry expect this syntax.

The available engines depend on what the framework supports. The one in the example is mod_perl 1.3 in the syntax of Catalyst and Gantry.

template_engine

Similar to engine, this specifies the template engine. Choices almost always include TT, but might also include Mason or other templaters depending on what your framework supports..

Other KEYWORDS

literal

This keyword applies to many backends at the app level and at some other levels. This keyword is special, because it expects a type keyword immediately before its values. For example:

literal SQL `CREATE...`;

It always instructs someone (the backend of type SQL in the example) to directly insert the backquoted string into its output, without so much as adjusting whitespace.

Backend types that should obey this statement are:

SQL      - for backends of type SQL
Location - for backends constructing apache confs or the like

The literal Location statement may also be used at the controller level.

no_gen

Applies to backend blocks in the config block, app blocks, controller blocks, and method blocks.

gen_from_string enforces the app level no_gen. If it has a true value only a warning is printed, nothing is generated. None of the backends are called.

gen_from_string also enforces no_gen on entire backends, if their config block has a true no_gen value.

The Control backend of your choice is responsible for enforcing no_gen at the controller and method levels.

not_for

Applies to tables and fields (although the latter only worked for Models at the time of this writing).

Each backend is responsible for enforcing not_for. It should mean that the field or table is ignored by the named backend type. Thus

table skip_model {
    not_for Model;
}

should generate as normal in SQL backends, but should be completely ignored for Models. The same should hold for fields marked not_for. But my SQL backends didn't do that when I wrote this, only the Models worked.

AUTHOR

Phil Crow <philcrow2000@yahoo.com>

COPYRIGHT and LICENSE

Copyright (C) 2005 by Phil Crow

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself, either Perl version 5.8.6 or, at your option, any later version of Perl 5 you may have available.

cpanm Bigtop

perl -MCPAN -e shell
install Bigtop