NAME
Plucene::Analysis::CJKAnalyzer - Analyzer for CJK texts
SYNOPSIS
# isa Plucene::Analysis::Analyzer
my Plucene::Analysis::CJKAnalyzer $wt
= Plucene::Analysis::CJKAnalyzer->new(@args);
DESCRIPTION
This is a text analyzer for analyzing CJK texts. Plucene does not support CJK texts natively. This module encodes terms in MIME::Base64 format to get around this problem. Texts are assumbed to be in UTF-8 encoding.
See "cjk.t" in t for more details.
METHODS
tokenstream
my Plucene::Analysis::CJKAnalyzer Plucene::Analysis::ChineseAnalyzer->new(@args);Creates a TokenStream which tokenizes all the text in the provided Reader.
SEE ALSO
Plucene::Analysis::CJKTokenizer
COPYRIGHT
Copyright (C) 2006 by Yung-chung Lin (a.k.a. xern) <xern@cpan.org>
This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself