Changes for version 0.13 - 2015-03-25

  • Removed referrer_url attribute from job class.
  • Changed scrape API significantly.
  • Added context option for scrape method.
  • Improved documentations.
  • Improved examples.

Modules

A web crawling framework for Perl
Single crawler job
Crawler queue base class
Crawler queue with memory
Scraper utitlities
Crawler specific featured user agent