NAME

Treex::Block::W2A::CS::Tokenize

VERSION

version 2.20151102

DESCRIPTION

Each sentence is split into a sequence of tokens. The default language-independent tokenization is used.

AUTHORS

David Mareček <marecek@ufal.mff.cuni.cz>

Ondřej Dušek <odusek@ufal.mff.cuni.cz>

COPYRIGHT AND LICENSE

Copyright © 2011 by Institute of Formal and Applied Linguistics, Charles University in Prague

This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself.