Why not adopt me?
NAME
Lingua::Stem::Snowball - Perl interface to Snowball stemmers.
SYNOPSIS
use Lingua::Stem::Snowball;
my @lang = stemmers();OO interface:
my $lang = 'english';
my $dict = Lingua::Stem::Snowball->new(lang => $lang);
my $locale = 'C';
my $dict = Lingua::Stem::Snowball->new(lang => $lang, locale => $locale);
my $lemm = $dict->stem($word);
my $lemm = $dict->stem($word, \$is_stemmed);
my $dict = Lingua::Stem::Snowball->new();
$dict->lang($lang);
$dict->locale($locale);
my $lemm = $dict->stem($word);
my @lemm = $dict->stem(\@words);Plain interface:
my $lemm = stem($lang, $word);
my $lemm = stem($lang, $word, $locale);
my $lemm = stem($lang, $word, $locale, \$is_stemmed);DESCRIPTION
This module provides unified perl interface to Snowball stemmers (http://snowball.tartarus.org) and virtually supports various languages. It's written using C for high performance and provides OO and plain interfaces.
The motivation of developing this module was to provide a generic access to stemming algorithms for OpenFTS project - full text search engine (http://openfts.sourceforge.net).
The module is very similar with Lingua::Stem. But Lingua::Stem is written in pure perl whereas Lingua::Stem::Snowball is an XS version of the snowball stemmers.
The following stemmers are available (as of Lingua::Stem 0.70):
|------------------------------|
| Language | L:S | L:S:S |
|------------------------------|
| English | y | y |
| French | y | y |
| Spanish | n | y |
| Portuguese | y | y |
| Italian | y | y |
| German | y | y |
| Dutch | n | y |
| Swedish | y | y |
| Norwegian | y | y |
| Danish | y | y |
| Russian | n | y |
| Finnish | n | y |
| Galician | y | n |
|------------------------------|Here is a little benchmark with examples files from the snowball distribution (with no cache):
|---------------------------------------------------|
| Language | Unique | Time (s) |
| | words | L:S:S | L:S:S | L:S | L:S:S |
| | | @ | $ | @ | $ |
|---------------------------------------------------|
| DA | 23829 | 0.5 | 1.1 | 7.3 | 14.2 |
| DE | 35033 | 0.9 | 1.9 | 64.3 | 73.5 |
| EN | 30428 | 0.7 | 1.5 | 2.5 | 8.8 |
| FR | 20403 | 0.6 | 1.1 | 182.7 | 188.0 |
| IT | 35494 | 1.0 | 2.0 | 345.6 | 350.2 |
| NO | 20628 | 0.4 | 1.0 | 14.3 | 20.6 |
| PT | 32016 | 0.8 | 1.7 | 405.6 | 414.8 |
| SV | 30623 | 0.0 | 0.5 | 15.9 | 25.6 |
|---------------------------------------------------|Here is the same benchmark with all unique words found in the bible:
|---------------------------------------------------|
| EN | 12718 | 0.3 | 0.7 | 1.0 | 3.6 |
|---------------------------------------------------|METHODS
- $dict = Lingua::Stem::Snowball->new
-
Creates a new instance of the stemmer.
The constructor takes hash style parameters. The following parameters are recognized:
lang: language (ISO code).
locale: locale.
- my $stemmed = $dict->stem($word)
-
Returns the stemmed word for $word.
- my @stemmed = $dict->stem(\@word)
-
Returns an array of the stemmed words contained in @word.
- $dict->lang([$lang])
-
Accessor for the lang parameter. If there is no stemmer for $lang, the language is not changed.
- $dict->locale([$locale])
-
Accessor for the locale parameter.
- stemmers()
-
Returns a list of all available languages with a stemmer.
REQUESTS & BUGS
Please report any requests, suggestions or bugs via the RT bug-tracking system at http://rt.cpan.org/ or email to bug-Lingua-Stem-Snowball\@rt.cpan.org.
http://rt.cpan.org/NoAuth/Bugs.html?Dist=Lingua-Stem-Snowball is the RT queue for Lingua::Stem::Snowball. Please check to see if your bug has already been reported.
COPYRIGHT
Copyright 2004
Currently maintained by Fabien Potencier, fabpot@cpan.org Original authors Oleg Bartunov, oleg@sai.msu.su, Teodor Sigaev, teodor@stack.net
This software may be freely copied and distributed under the same terms and conditions as Perl.
Snowball files and stemmers are covered by the BSD license.
SEE ALSO
http://snowball.tartarus.org, Lingua::Stem