NAME
WWW::Crawler::Mojo::UserAgent - Crawler specific featured user agent
SYNOPSIS
my $ua = WWW::Crawler::Mojo::UserAgent->new;
$ua->keep_credentials(1);
$ua->credentials(
'http://example.com:8080' => 'jamadam:password1',
'http://example2.com:8080' => 'jamadam:password2',
);
my $tx = $ua->get('http://example.com/');
say $tx->req->url # http://jamadam:passowrd@example.com/
if ($ua->active_conn < $max_conn) {
$ua->get(...);
}
if ($ua->active_host($url) < $max_conn_per_host) {
$ua->get(...);
}DESCRIPTION
This class inherits Mojo::UserAgent and adds credential storage and active connection counter.
ATTRIBUTES
WWW::Crawler::Mojo::UserAgent inherits all attributes from Mojo::UserAgent.
active_conn
A number of current connections.
$bot->active_conn($bot->active_conn + 1);
say $bot->active_conn;active_conn_per_host
A number of current connections per host.
$bot->active_conn_per_host($bot->active_conn_per_host + 1);
say $bot->active_conn_per_host;keep_credentials
Sets true to activate the feature. Defaults to 1.
$ua->keep_credentials(1);METHODS
WWW::Crawler::Mojo::UserAgent inherits all methods from Mojo::UserAgent.
active_host
Maintenances the numbers of active connections.
$ua->active_host($url, 1);
$ua->active_host($url, -1);
my $amount = $ua->active_host($url);credentials
Stores credentials.
$ua->credentials(
'http://example.com:8080' => 'jamadam:password1',
'http://example2.com:8080' => 'jamadam:password2',
);new
Constructer.
$ua = WWW::Crawler::Mojo::UserAgent->new;AUTHOR
Keita Sugama, <sugama@jamadam.com>
COPYRIGHT AND LICENSE
Copyright (C) Keita Sugama.
This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.