Changes for version 0.21 - 2017-01-17

  • Fix a bug where url including white spaces haven't been treated propery.
  • Form submition now includes passwords and dates.
  • Started Appveyor to test against windows.

Modules

A web crawling framework for Perl
Single crawler job
Crawler queue base class
Crawler queue with memory
Crawler queue with Mysql [EXPERIMENTAL]
Scraper utitlities
Crawler specific featured user agent