NAME

CPAN::Search::Lite::Index - set up or update database tables.

SYNOPSIS

my $index = CPAN::Search::Lite::Index->new(config => 'cpan.conf', setup => 1);
$index->index();

DESCRIPTION

This is the main module used to set up or update the database tables used to store information from the CPAN and ppm indices. The creation of the object

my $index = CPAN::Search::Lite::Index->new(%args);

accepts three arguments:

config => /path/to/config.conf

This argument specifies where to find the configuration file used to determine the remaining options. In lieu of this option, the environment variable CSL_CONFIG_FILE pointing to the configuration file may be specified.
setup => 1

This (optional) argument specifies that the database is being set up. Any existing tables will be dropped.
reindex => value

This (optional) argument specifies distribution names that one would like to reindex in an existing database. These may be specified as either a scalar, for a single distribution, or as an array reference for a list of distributions.

CONFIGURATION

Most of the options used to control the behaviour of the indexing are contained in a configuration file. An example of the format of such a file is

[CPAN]

CPAN = /var/ftp/pub/CPAN
pod_root = /usr/local/POD
html_root = /usr/local/httpd/htdocs/CPAN

[DB]

db = pause
user = sarah
passwd = lianne

[WWW]

css = cpan.css
up_img = up.gif
tt2 = /usr/local/tt2
geoip = /usr/local/share/geoip/cpan.txt

This consists of 3 sections.

CPAN

This is associated with various things related to CPAN.

CPAN = /var/ftp/pub/CPAN

This specifies the root directory of the local CPAN mirror, if this exists, or the location where the CPAN index files will be downloaded and kept, if the no_mirror option is specified.
pod_root = /usr/local/POD

This specifies where the extracted pod files from a distribution will be kept. A subdirectory dist_name under this directory will be created corresponding to the name of the distribution.
pod_only = 1

This specifies that, if the module files are to be extracted, fetch only those that contain pod.
split_pod = 1

This specifies that, if the module files are to be extracted, when generating the html pages create two pages for each module: one containing just the documentation, and the other containing the code run through Perl::Tidy. For a module such as Foo::Bar, the documentation will be saved as a file Foo/Bar.html, while the sources will be saved as Foo/Bar.pm.html.
html_root = /usr/local/httpd/htdocs/CPAN

This specifies where the html files created from the pod files will be kept. A subdirectory dist_name under this directory will be created corresponding to the name of the distribution.
ignore = some_dist_name_to_ignore

This specifies a name of a distribution (without a version number) to ignore in indexing. This option may be given a number of times to specify an array of values, or may be specified as
```
ignore = <<EOL
Module-CPANTS-asHash
CORBA-IDL
EOL
```
This array of values (which may include regular expressions) is joined together as
```
$pat = join '|', @ignore_dists
```
and if the distribution name matches
```
$dist_name =~ /^($pat)$/
```
the distribution is ignored.
no_mirror = 1

This specifies that a local CPAN mirror isn't available, and as such no pod or html files will be extracted nor created.
no_ppm = 1

This can be used to signal to not gather information on Win32 ppm packages from the repositories specified in $repositories of CPAN::Search::Lite::Util.
remote_mirror = http://cpan.wherever.edu

If no_mirror is specified, the value of remote_mirror will be used to fetch the CPAN indices. If not given, http://www.cpan.org will be used.
multiplexer = http://cpan.redirect.edu/cpan

This can be used to specify a multiplexer to redirect downloads to nearby CPAN mirrors. See, for example, Apache::GeoIP for one implementation.
cat_threshold = 0.99

Many modules do not have a category (chapter) associated with them. In such cases, when populating the database, the AI::Catgorizer module is used to guess which category should be assigned to such a module, based on available information for those modules that do have a category. The value of cat_threshold is used to determine if the guessed category should be accepted (a perfect match has a score of 1, and no match has 0). If no such value is given, a default of 0.995 is used.
no_cat = 1

Set no_cat equal to a true value if you don't want AI::Categorizer to try categorizing modules which don't have a category assigned.

DB

This is used to store connection information to the database used to populate the tables.

db = pause

This is the name of the database used. It is assumed here that the database has already been created, and that appropriate read, write, update, create, and delete permissions for the user specified below have already been granted.
user = sarah

This is the user under which to connect to the database.
passwd = lianne

This is the password to use for the user.

WWW

This is used for various information related to a web interface.

css = cpan.css

If specified, this will be used as the css file when generating the html files from the pod files of the distribution. It is assumed this file appears directly beneath the html_root of the CPAN section.
up_img = up.gif

If specified, this will be used as an image in the generated html files linking each section to the top-most index. If not specifed, the text __top will be used. It is assumed this image appears directly beneath the html_root of the CPAN section.
tt2 = /usr/local/tt2

This gives the location of the Template-Toolkit pages used to provide a web interface. This is used to place a file mirror_list (extracted from $CPAN/MIRRORED.BY) containing a list of CPAN mirrors.
geoip = /usr/local/share/geoip/cpan.txt

If the module Geo::IP or Apache::GeoIP is used to provide a redirection service to a nearby CPAN mirror based on the location of origin, this file will be created to provide the necessary country of origins of the CPAN mirrors.

DETAILS

Calling

$index->index();

will start the indexing procedure. Various messages detailing the progress will written to STDOUT, which by default will be captured into a file cpan_search_log.dddddddddd, where the extension is the time that the method was invoked. Passing index an argument of log => log_file will save these messages into log_file. Error messages are not captured, and will appear in STDERR.

The steps of the indexing procedure are as follows.

fetch index data

If the no_mirror option is specified, the necessary CPAN index files $CPAN/MIRRORED.BY, $CPAN/indices/ls-lR.gz, $CPAN/authors/01mailrc.txt.gz, $CPAN/modules/02packages.details.txt.gz, and $CPAN/modules/03modlist.data.gz will be fetched from the CPAN mirror specified by the $cpan variable at the beginning of CPAN::Search::Lite::Index. If you are using this option, it is recommended to use the same CPAN mirror with subsequent updates, to ensure consistency of the database. As well, the information on the locations of the CPAN mirrors used for Template-Toolkit and GeoIP is written.
get index information

Information from the CPAN indices and, if desired, the ppm repositories is extracted. This is done through CPAN::Search::Lite::Info (for the CPAN indices) and CPAN::Search::Lite::PPM (for the ppm repositories).
get state information

Unless the setup argument within the new method of CPAN::Search::Lite::Index is specified, this will get information on the state of the database through CPAN::Search::Lite::State. A comparision is then made between this information and that gathered from the CPAN indices, and if there's a discrepency in some items, those items are marked for either insertion, updating, or deletion, as appropriate.
extract files

Unless the no_mirror option is specified, this will extract, through CPAN::Search::Lite::Extract, the available pod sections of files of a distribution, placing them under a subdirectory dist_name (corrsponding to the name of the distribution) under the specified pod_root in the configuration file. pod2html is then run on them, with the results placed under dist_name of html_root. Also, a README, Changes, INSTALL, and META.yml file, if present, will be copied over into dist_name under pod_root. Finally, information on the prerequisites of the distribution, and distribution and module descriptions, if available and needed, is extracted.
populate the database

At this stage the gathered information is used to populate the database, through CPAN::Search::Lite::Populate, either inserting new items, updating existing ones, or deleting obsolete items.

As well, unless the no_mirror option is specified, the html files created under html_root will be edited to adjust the links to module files. This is necessary because when the html files are created no cache is used (in order to maintain consistency between updates), and consequently links to packages outside of a given package may be incorrect. This is fixed by querying the database to see what module documentation is actually present, and then adjusting the links in the html files accordingly (or removing a link if the indicated documentation is missing).

COPYRIGHT

To install CPAN::Search::Lite, copy and paste the appropriate command in to your terminal.

cpanm

cpanm CPAN::Search::Lite

CPAN shell

perl -MCPAN -e shell
install CPAN::Search::Lite

For more information on module installation, please visit the detailed CPAN module installation guide.

	Global
`s`	Focus search bar
`?`	Bring up this help dialog

	GitHub
`g` `p`	Go to pull requests
`g` `i`	go to github issues (only if github is preferred repository)

	POD
`g` `a`	Go to author
`g` `c`	Go to changes
`g` `i`	Go to issues
`g` `d`	Go to dist
`g` `r`	Go to repository/SCM
`g` `s`	Go to source
`g` `b`	Go to file browse

	Search terms
module: (e.g. module:Plugin)
distribution: (e.g. distribution:Dancer auth)
author: (e.g. author:SONGMU Redis)
version: (e.g. version:1.00)