NAME

SQL::Tokenizer - A simple SQL tokenizer.

VERSION

SYNOPSIS

use SQL::Tokenizer;

my $query = q{SELECT 1 + 1};
my @tokens = SQL::Tokenizer->tokenize($query);

# @tokens contains ('SELECT', ' ', '1', ' ', '+', ' ', '1')

DESCRIPTION

SQL::Tokenizer is a simple tokenizer for SQL queries. It does not claim to be a parser or query verifier. It just creates sane tokens from a valid SQL query.

It supports SQL with comments like:

-- This query is used to insert a message into
-- logs table
INSERT INTO log (application, message) VALUES (?, ?)

Also supports '', "" and \' escaping methods, so tokenizing queries like the one below should not be a problem:

INSERT INTO log (application, message)
VALUES ('myapp', 'Hey, this is a ''single quoted string''!')

API

tokenizer

my @tokens= SQL::Tokenizer->tokenize($query);
my $tokens= SQL::Tokenizer->tokenize($query);

$tokens= SQL::Tokenizer->tokenize( $query, $remove_white_tokens );

This is the only available method. It receives a SQL query, and returns an array of tokens if called in list context, or an arrayref if called in scalar context.

If $remove_white_tokens is true, white spaces only tokens will be removed from result.

ACKNOWLEDGEMENTS

Evan Harris, for implementing Shell comment style and SQL operators.

AUTHOR

Copyright (c) 2007, Igor Sutton Lopes "<IZUT@cpan.org>". All rights reserved.

This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself.