NAME

App::Wubot::Util::WebFetcher - fetch content from the web

VERSION

version 0.5.0

SYNOPSIS

use App::Wubot::Util::WebFetcher;

has 'fetcher' => ( is  => 'ro',
                   isa => 'App::Wubot::Util::WebFetcher',
                   lazy => 1,
                   default => sub {
                       return App::Wubot::Util::WebFetcher->new();
                   },
               );

my $content;
eval {                          # try
    $content = $self->fetcher->fetch( $config->{url}, $config );
    1;
} or do {                       # catch
    my $error = $@;
    my $subject = "Request failure: $error";
    $self->logger->error( $self->key . ": $subject" );
    return { cache => $cache, react => { subject => $subject } };
};

DESCRIPTION

Fetch data from a URL using LWP::UserAgent.

This utility class is designed to be used by wubot plugins in the classes App::Wubot::Plugin and App::Wubot::Reactor.

SUBROUTINES/METHODS

$obj->fetch( $url, $config );

Fetches content from the specified URL.

If the fetch attempt fails, then this method will die with the error message. Plugins that wish to use this library should wrap the fetch() method in an eval block (see the example above).

The data fetched will be passed to utf8::encode().

The 'config' may contain the following settings:

timeout

Number of seconds before the fetch times out. Defaults to 20 seconds.

agent

The user agent string. Defaults to 'Mozilla/6.0'.

user

User id for basic auth. See also 'pass'

pass

Password for basic auth. See also 'user'.

proxy

The proxy URL.

decompress

If true, sets the Accept-Encoding using HTTP::message::decodable.

ignore_cert_errors

If true, do not perform SSL certificate validation for https URLs.

ca_file

If you have a custom SSL CA file for this monitor, set the path using this parameter. If you want to use the same ca file for all monitors, it is probably better to set the path using either the environment variable PERL_LWP_SSL_CA_FILE or HTTPS_CA_FILE.

SOCKS PROXIES

If you are using a socks proxy, set the 'proxy' to something like this:

socks://localhost:1080/

You can create a compatible socks proxy using ssh:

ssh -N -D 1080 somehost

You may also need to install:

LWP::Protocol::socks

If LWP::Protocol::socks is not properly installed, you may see an error message such as:

501 Protocol scheme ''socks'' is not supported.