Changes for version 0.23 - 2019-04-02

  • Now enqueue methods returns the actual jobs
  • Now it's breadth-first on memory capacity control

Modules

A web crawling framework for Perl
Single crawler job
Crawler queue base class
Crawler queue with memory
Crawler queue with Mysql [EXPERIMENTAL]
Scraper utitlities
Crawler specific featured user agent