/ 
Pistachio-0.10
River stage zero No dependents
/ Pistachio::Tokenizer

NAME

Pistachio::Tokenizer - provides iterator(), which turns source code text into a Pistachio::Token iterator

VERSION

version 0.10

SYNOPSIS

use Pistachio::Tokenizer;
my $tokenizer = Pistachio::Tokenizer->new('Perl5');

my $scalar_ref = \"use strict; ...;";
my $it = $tokenizer->iterator($scalar_ref);

while (my $token = $it->()) {
    print $token->type, ': ', $token->value, "\n";
}

AUTHOR

Joel Dalley <joeldalley@gmail.com>

COPYRIGHT AND LICENSE

This software is copyright (c) 2014 by Joel Dalley.

This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.