NAME
Treex::Block::W2A::CS::Tokenize
VERSION
version 2.20151102
DESCRIPTION
Each sentence is split into a sequence of tokens. The default language-independent tokenization is used.
AUTHORS
David Mareček <marecek@ufal.mff.cuni.cz>
Ondřej Dušek <odusek@ufal.mff.cuni.cz>
COPYRIGHT AND LICENSE
Copyright © 2011 by Institute of Formal and Applied Linguistics, Charles University in Prague
This module is free software; you can redistribute it and/or modify it under the same terms as Perl itself.