The London Perl and Raku Workshop takes place on 26th Oct 2024. If your company depends on Perl, please consider sponsoring and/or attending.
 / 
HTML-WikiConverter-0.52
River stage two • 22 direct dependents • 24 total dependents
/ HTML::WikiConverter::Dialects

NAME

HTML::WikiConverter::Dialects - How to add a dialect

SYNOPSIS

  # In your dialect module:

  package HTML::WikiConverter::MySlimWiki;
  use HTML::WikiConverter -dialect;

  rule b => { start => '**', end => '**' };
  rule i => { start => '//', end => '//' };
  rule strong => { alias => 'b' };
  rule em => { alias => 'i' };
  rule hr => { replace => "\n----\n" };

  # In a nearby piece of code:

  package main;
  use Test::More tests => 5;

  my $wc = new HTML::WikiConverter(
    dialect => 'MySlimWiki'
  );

  is( $wc->html2wiki( '<b>text</b>' ), '**text**', b );
  is( $wc->html2wiki( '<i>text</i>' ), '//text//', i );
  is( $wc->html2wiki( '<strong>text</strong>' ), '**text**', 'strong' );
  is( $wc->html2wiki( '<em>text</em>' ), '//text//', 'em' );
  is( $wc->html2wiki( '<hr/>' ), '----', 'hr' );

DESCRIPTION

HTML::WikiConverter (or H::WC, for short) is an HTML to wiki converter. It can convert HTML source into a variety of wiki markups, called wiki "dialects". This manual describes how you to create your own dialect to be plugged into HTML::WikiConverter.

DIALECTS

Each dialect has a separate dialect module containing rules for converting HTML into wiki markup specific for that dialect. Currently, all dialect modules are in the HTML::WikiConverter:: package space and subclass HTML::WikiConverter. For example, the MediaWiki dialect module is HTML::WikiConverter::MediaWiki, while PhpWiki's is HTML::WikiConverter::PhpWiki.

From now on, I'll be using the terms "dialect" and "dialect module" interchangeably.

Subclassing

To interface with H::WC, dialects need to subclass it. Because you'll probably be wanting the rule() and attribute() functions as well, subclassing and importing these functions is done in a single step:

  use HTML::WikiConverter -dialect;

This will add HTML::WikiConverter to your dialect's @ISA and will import the attribute() and rule() functions into your dialect's package.

Conversion rules

Dialects guide H::WC's conversion process with a set of rules that define how HTML elements are turned into their wiki counterparts. Each rule corresponds to an HTML tag (including nonstandard tags), and there may be any number of rules. Rules are added with the rule() function that was imported when you subclassed H::WC (see above).

The syntax for rule() is as follows:

    rule $tag => \%subrules;

where $tag is the name of the HTML tag (eg, 'b', 'em', etc.) and %subrules contains subrules that specify how that tag will be converted.

Supported subrules

The following subrules are recognized:

  start
  end

  preserve
  attributes
  empty

  replace
  alias

  block
  line_format
  line_prefix

  trim

A simple example

The following rules could be used for a dialect that uses *asterisks* for bold and _underscores_ for italic text:

  rule b => { start => '*', end => '*' };
  rule i => { start => '_', end => '_' };

Aliases

To add <strong> and <em> as aliases of <b> and <i>, use the alias subrule:

  rule b => { start => '*', end => '*' };
  rule i => { start => '_', end => '_' };

  rule strong => { alias => 'b' };
  rule em => { alias => 'i' };

(The alias subrule cannot be used with any other rule.)

Blocks

Many dialects separate paragraphs and other block-level elements with a blank line. To indicate this, use the block subrule:

  rule p => { block => 1 };

(To better support nested block elements, if a block elements are nested inside each other, blank lines are only added to the outermost element.)

Line formatting

Many dialects require that the text of a paragraph be contained on a single line of text. Or perhaps that a paragraph cannot contain any newlines. These options can be specified using the line_format subrule, which can be assigned the value "single", "multi", or "blocks".

If the element must be contained on a single line, then the line_format subrule should be "single". If the element can span multiple lines, but there can be no blank lines contained within, then it should be "multi". If blank lines (which delimit blocks) are allowed, then use "blocks". For example, paragraphs are specified like so in the MediaWiki dialect:

  rule p => { block => 1, line_format => 'multi', trim => 'both' };

Trimming whitespace

The trim subrule specifies whether leading or trailing whitespace (or both) should be stripped from the element. To strip leading whitespace only, use "leading"; for trailing whitespace, use "trailing"; for both, use the aptly named "both"; for neither (the default), use "none".

Line prefixes

Some elements require that each line be prefixed with a particular string. This is specified with the line_prefix subrule. For example, preformatted text in MediaWiki is prefixed with a space:

  rule pre => { block => 1, line_prefix => ' ' };

(There is actually a known bug [#14527] with the processing of whitespace that causes line prefixes to be removed from the wiki markup after conversion. I'm working on a fix for this.)

Replacement

In some cases, conversion from HTML to wiki markup is as simple as string replacement. To replace a tag and its contents with a particular string, use the replace subrule. For example, in PhpWiki, three percent signs '%%%' represents a linebreak <br />, hence:

  rule br => { replace => '%%%' };

(The replace subrule cannot be used with any other rule.)

Preserving HTML tags

Some dialects allow a subset of HTML in their markup. While H::WC ignores unknown (ie, nonstandard) HTML tags by default, you may specify that they are preserved using the preserve subrule. For example, to allow <font> tag in wiki markup:

  rule font => { preserve => 1 };

Preserved tags may also specify a list of attributes that may likewise pass-through from HTML to wiki markup. This is done with the attributes subrule:

  rule font => { preserve => 1, attributes => [ qw/ font size / ] };

(The attributes subrule can only be used if the preserve subrule is also present.)

Some HTML elements have no content (e.g. line breaks, images), and might need to be preserved in a more XHTML-friendly way. To indicate that a preserved tag should have no content, use the empty subrule. This will cause the element to be replaced with "<tag />", with no end tag. For example, MediaWiki handles line breaks like so:

  rule br => {
    preserve => 1,
    attributes => qw/ id class title style clear /,
    empty => 1
  };

This will convert, e.g., "<br clear='both'>" into "<br clear='both' />". Without specifying the empty subrule, this would be converted into the perhaps undesirable "<br clear='both'></br>".

(The empty subrule can only be used if the preserve subrule is also present.)

Dynamic subrules

Instead of simple strings, you may use coderefs as values for the start, end, replace, and line_prefix subrules. If you do, the code will be called when the subrule is applied, and will be passed three arguments: the current H::WC object, the current HTML::Element node being operated on, and a reference to the hash containing the dialect's subrules associated with elements of that type.

For example, MoinMoin handles lists like so:

  rule ul => { line_format => 'multi', block => 1, line_prefix => '  ' };
  rule li => { start => \&_li_start, trim => 'leading' };
  rule ol => { alias => 'ul' };

And then defines _li_start():

  sub _li_start {
    my( $self, $node, $subrules ) = @_;
    my $bullet = '';
    $bullet = '*'  if $node->parent->tag eq 'ul';
    $bullet = '1.' if $node->parent->tag eq 'ol';
    return "\n$bullet ";
  }

This ensures that every unordered list item is prefixed with * and every ordered list item is prefixed with 1., required by the MoinMoin formatting rules. It also ensures that each list item is on a separate line and that there is a space between the prefix and the content of the list item.

Subrule validation

Certain subrule combinations are not allowed. For example, the replace and alias subrules cannot be combined with any other subrules, and attributes can only be specified alongside preserve. Invalid subrule combinations will trigger a fatal error when the H::WC object is instantiated.

Dialect attributes

H::WC's constructor accepts a number of attributes that help determine how conversion takes place. Dialects can alter these attributes or add their own by using the attribute() function, which (like rule()) was imported when H::WC was subclassed (see above). Its syntax is:

  attribute $attr => \%spec;

where $attr is the name of the attribute and %spec is a Params::Validate specification for the attribute.

For example, to add a boolean attribute called camel_case which is disabled by default:

  attribute camel_case => { default => 0 };

Attributes defined liks this are given accessor and mutator methods via Perl's AUTOLOAD mechanism, so you can later say:

  my $ok = $wc->camel_case; # accessor
  $wc->camel_case(0); # mutator

You may override the default H::WC attributes using this mechamism. For example, the PbWiki dialect requires that the base_uri element be defined but H::WC doesn't require it by default (all attributes are optional by default). Thus the PbWiki dialect could override this using:

  attribute base_uri => { optional => 0 };

Preprocessing

The first step in converting HTML source to wiki markup is to parse the HTML into a syntax tree using HTML::TreeBuilder. It is often useful for dialects to preprocess the tree prior to converting it into wiki markup. Dialects that need to preprocess the tree define a preprocess_node method that will be called on each node of the tree (traversal is done in pre-order). As its only argument the method receives the current HTML::Element node being traversed. It may modify the node or decide to ignore it. The return value of the preprocess_node method is discarded.

Built-in preprocessors

Because they are commonly needed, two preprocessing steps are automatically carried out by HTML::WikiConverter, regardless of the dialect: 1) relative URIs in images and links are converted to absolute URIs (based upon the base_uri parameter), and 2) ignorable text (e.g. between a </td> and <td>) is discarded.

H::WC also provides additional preprocessing steps that may be explicitly enabled by dialect modules.

strip_aname

Removes from the HTML input any anchor elements that do not contain an href attribute.

caption2para

Removes table captions and reinserts them as paragraphs before the table.

Dialects may apply these optional preprocessing steps by calling them as methods on the dialect object inside preprocess_node. For example:

  sub preprocess_node {
    my( $self, $node ) = @_;
    $self->strip_aname($node);
    $self->caption2para($node);
  }

Postprocessing

Once the work of converting HTML, it is sometimes useful to postprocess the resulting wiki markup. Postprocessing can be used to clean up whitespace, fix subtle bugs introduced in the markup during conversion, etc.

Dialects that want to postprocess the wiki markup should define a postprocess_output object method that will be called just before thehtml2wiki method returns to the client. The method will be passed a single argument, a reference to the wiki markup. It may modify the wiki markup that the reference points to. Its return value is discarded.

For example, to replace a series of line breaks with a pair of newlines, a dialect might implement this:

  sub postprocess_output {
    my( $self, $outref ) = @_;
    $$outref =~ s/<br>\s*<br>/\n\n/gs;
  }

(This example assumes that HTML line breaks were replaced with <br> in the wiki markup.)

Dialect utility methods

HTML::WikiConverter defines a set of utility methods that dialect modules may find useful.

get_elem_contents
  my $wiki = $wc->get_elem_contents( $node );

Converts the contents of $node into wiki markup and returns the resulting wiki markup.

get_wiki_page
  my $title = $wc->get_wiki_page( $url );

Attempts to extract the title of a wiki page from the given URL, returning the title on success, undef on failure. If wiki_uri is empty, this method always return undef. See "ATTRIBUTES" in HTML::WikiConverter for details on how the wiki_uri attribute is interpreted.

is_camel_case
  my $ok = $wc->is_camel_case( $str );

Returns true if $str is in CamelCase, false otherwise. CamelCase-ness is determined using the same rules that Kwiki's formatting module uses.

get_attr_str
  my $attr_str = $wc->get_attr_str( $node, @attrs );

Returns a string containing the specified attributes in the given node. The returned string is suitable for insertion into an HTML tag. For example, if $node refers to the HTML

  <style id="ht" class="head" onclick="editPage()">Header</span>

and @attrs contains "id" and "class", then get_attr_str will return 'id="ht" class="head"'.

_attr
  my $value = $wc->_attr( $name );

Returns the value of the named attribute. This is rarely needed since you can access attribute values by treating the attribute name as a method (eg, $wc->$name). This low-level method of accessing attributes is useful when you need to override an attribute's accessor/mutator method, as in:

  attribute my_attr => { default => 1 };

  sub my_attr {
    my( $wc, $name, $new_value ) = @_;
    # do something special
    return $wc->_attr( $name => $new_value );
  }

AUTHOR

David J. Iberri <diberri@cpan.org>

COPYRIGHT

COPYRIGHT & LICENSE

Copyright 2006 David J. Iberri, all rights reserved.

This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.