NAME

WWW::Search::Scraper - framework for scraping results from search engines.

NOTE: You can find a full description of the Scraper framework in WWW::Search::Scraper::ScraperPOD.pm.

SYNOPSIS

use WWW::Search::Scraper;
$scraper = new WWW::Search::Scraper('engineName', { 'query' => $queryString });
$scraper->GetRequest->$fieldName($fieldValue);    
$response = $scraper->next_response();
print $response->$fieldName();

DESCRIPTION

NOTE: You can find a full description of the Scraper framework in WWW::Search::Scraper::ScraperPOD.pm.

"Scraper" is a framework for issuing queries to a search engine, and scraping the data from the resultant multi-page responses, and the associated detail pages.

As a framework, it allows you to get these results using only slight knowledge of HTML and Perl. (All you need to know you can learn by reading WWW::Search::Scraper::ScraperPOD.pm.)

A Perl script, "Scraper.pl", uses Scraper.pm to investigate the "advanced search page" of a search engine, issue a user specified query, and parse the results. (Scraper.pm can be used by itself to support more elaborate searching Perl scripts.) Scraper.pl and Scraper.pm have enough intelligence to figure out how to interpret the search page and its results.

MAJOR FEATURES

NOTE: You can find a full description of the Scraper framework in WWW::Search::Scraper::ScraperPOD.pm.

Framing

A simple opcode based language makes describing the results and details pages of new engines easy, and adapting to occasional changes in an existing engine's format simple.

Canonical Requests

A common Request container makes multiple search engine searches easy to implement, and automatically adapts to changes.

Canonical Response

A common Response container makes interpretation of results common among all search engines possible. Also adapts easily to changes.

Post-filtering

Post-filtering provides a powerful client-based extension of the search capabilities to all search engines.

AUTHOR

Glenn Wood, http://search.cpan.org/search?mode=author&query=GLENNWOOD.

COPYRIGHT

Copyright (C) 2001-2002 Glenn Wood. All rights reserved. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.