SYNOPSIS
use WebService::LOC::CongRec::Crawler;
use Log::Log4perl;
Log::Log4perl->init_once('log4perl.conf');
$crawler = WebService::LOC::CongRec::Crawler->new();
$crawler->congress(107);
$crawler->oldest(1);
$crawler->goForth();ATTRIBUTES
- congress
-
The numbered congress to be fetched. If this is not given, the current congress is fetched.
- issuesRoot
-
The root page for Daily Digest issues.
Breadcrumb path: Library of Congress > THOMAS Home > Congressional Record > Browse Daily Issues
- issues
-
A hash of issues: %issues{year}{month}{day}{section}
- mech
-
A WWW::Mechanize object with state that we can use to grab the page from Thomas.
- oldest
-
Boolean attribute specifying that pages are visited from earliest to most recent.
The default is 0 - that is visit most recent first.
METHODS
goForth()
$crawler->goForth();
$crawler->goForth(process => \&process_page);
$crawler->goForth(start => $x);
$crawler->goForth(end => $y);Start crawling from the Daily Digest issues page, i.e. http://thomas.loc.gov/home/Browse.php?&n=Issues
Also, for a specific congress, where NUM is congress number: http://thomas.loc.gov/home/Browse.php?&n=Issues&c=NUM
Returns the total number of pages grabbed.
Accepts an optional processing function to perform for each page.
Accpets optional page counter start and end ranges. If neither are given, or given as zero, crawing starts from the beginning and goes until all pages are visited.
parseRoot(Str $content)
Parse the the root of an issue an fill our hash of available issues