/ 
Resource-Silo-0.08
River stage zero No dependents
/ Resource::Silo

NAME

Resource::Silo - lazy declarative resource container for Perl.

DESCRIPTION

We assume the following setup:

  • The application needs to access multiple resources, such as configuration files, databases, queues, service endpoints, credentials, etc.

  • The application has helper scripts that don't need to initialize all the resources at once, as well as a test suite where accessing resources is undesirable unless a fixture or mock is provided.

  • The resource management has to be decoupled from the application logic where possible.

And we propose the following solution:

  • All available resources are declared in one place and encapsulated within a single container.

  • Such container is equipped with methods to access resources, as well as an exportable prototyped function for obtaining the one and true instance of it (a.k.a. optional singleton).

  • Every class or script in the project accesses resources through this container and only through it.

SYNOPSIS

The default mode is to create a one-off container for all resources and export if into the calling class via silo function.

package My::App;
use Resource::Silo;

use DBI;
use YAML::LoadFile;
...

resource config => sub { LoadFile( ... ) };
resource dbh    => sub {
    my $self = shift;
    my $conf = $self->config->{dbh};
    DBI->connect( $conf->{dsn}, $conf->{user}, $conf->{pass}, { RaiseError => 1 } );
};
resource queue  => sub { My::Queue->new( ... ) };
...

my $statement = silo->dbh->prepare( $sql );
my $queue = silo->queue;

For more complicated projects, it may make more sense to create a dedicated class for resource management:

# in the container class
package My::Project::Res;
use Resource::Silo -class;      # resource definitions will now create
                                # eponymous methods in My::Project::Res

resource foo => sub { ... };    # declare resources as in the above example
resource bar => sub { ... };

1;

# in all other modules/packages/scripts:

package My::Project;
use My::Project::Res qw(silo);

silo->foo;                      # obtain resources
silo->bar;

My::Project::Res->new;          # separate empty resource container

EXPORT

The following functions will be exported into the calling module, unconditionally:

  • silo - a singleton function returning the resource container. Note that this function will be created separately for every calling module, and needs to be re-exported to be shared.

  • resource - a DSL for defining resources, their initialization and properties. See below.

Additionally, if the -class argument was added to the use line, the following things happen:

  • Resource::Silo::Container and Exporter are added to @ISA;

  • silo function is added to @EXPORT and thus becomes re-exported by default;

  • calling resource creates a corresponding method in this package.

resource

resource 'name' => sub { ... };
resource 'name' => %options;

%options may include:

init => sub { $self, $name, [$argument] }

A coderef to obtain the resource. Required, unless literal or class are specified.

If the number of arguments is odd, the last one is popped and considered to be the init function.

literal => $value

Replace initializer with sub { $value }.

In addition, derived flag is set, and an empty dependencies list is implied.

argument => sub { ... } || qr( ... )

If specified, assume that the resource in question may have several instances, distinguished by a string argument. Such argument will be passed as the 3rd parameter to the init function.

This may be useful e.g. for DBIx::Class result sets, or for Redis::Namespace.

A regular expression will always be anchored to match the whole string. A function must return true for the parameter to be valid.

If the argument is omitted, it is assumed to be an empty string.

Example:

use Resource::Silo;
use Redis;
use Redis::Namespace;

resource real_redis => sub { Redis->new };

my %known_namespaces = (
    users    => 1,
    sessions => 1,
    counters => 1,
);

resource redis => argument => sub { $known_namespaces{ +shift } },
    init => sub {
        my ($self, $name, $ns) = @_;
        Redis::Namespace->new(
            redis     => $self->real_redis,
            namespace => $ns,
        );
    };

derived => 1 | 0

Assume that resource can be derived from its dependencies, or that it introduces no extra side effects compared to them.

This also naturally applies to resources with pure initializers, i.e. those having no dependencies and adding no side effects on top.

Examples may be Redis::Namespace built on top of a Redis handle or DBIx::Class built on top of DBI connection.

Derivative resources may be instantiated even in locked mode, as they would only initialize if their dependencies have already been initialized or overridden.

See "lock" in Resource::Silo::Container.

ignore_cache => 1 | 0

If set, don't cache resource, always create a fresh one instead. See also "fresh" in Resource::Silo::Container.

preload => 1 | 0

If set, try loading the resource when silo->ctl->preload is called. Useful if you want to throw errors when a service is starting, not during request processing.

cleanup => sub { $resource_instance }

Undo the init procedure. Usually it is assumed that the resource will do it by itself in the destructor, e.g. that's what a DBI connection would do. However, if it's not the case, or resources refer circularly to one another, a manual "destructor" may be specified.

It only accepts the resource itself as an argument and will be called before erasing the object from the cache.

See also fork_cleanup.

cleanup_order => $number

The higher the number, the later the resource will get destroyed.

The default is 0, negative numbers are also valid, if that makes sense for you application (e.g. destroy $my_service_main_object before the resources it consumes).

fork_cleanup => sub { $resource_instance }

Like cleanup, but only in case a change in process ID was detected. See "FORKING"

This may be useful if cleanup is destructive and shouldn't be performed twice.

dependencies => \@list

If specified, only allow resources from the list to be fetched in the initializer.

This parameter has a different meaning if class parameter is in use (see below).

class => 'Class::Name'

Turn on Spring-style dependency injection. This forbids init and argument parameters and requires dependencies to be a hash.

The dependencies' keys become the arguments to Class::Name->new, and the values format is as follows:

  • argument_name => resource_name

    Use a resource without parameter;

  • argument_name => [ resource_name => argument ]

    Use a parametric resource;

  • resource_name => 1

    Shorthand for resource_name => resource_name;

  • name => \$literal_value

    Pass $literal_value to the constructor as is.

So this:

resource foo =>
    class           => 'My::Foo',
    dependencies    => {
        dbh     => 1,
        redis   => [ redis => 'session' ],
        version => \3.14,
    };

Is roughly equivalent to:

resource foo =>
    dependencies    => [ 'dbh', 'redis' ],
    init            => sub {
        my $c = shift;
        require My::Foo;
        My::Foo->new(
            dbh     => $c->dbh,
            redis   => $c->redis('session'),
            version => 3.14,
        );
    };

require => 'Module::Name' || \@module_list

Load module(s) specified before calling the initializer.

This is exactly the same as calling require 'Module::Name' in the initializer itself except that it's more explicit.

silo

A re-exportable singleton function returning one and true Resource::Silo::Container instance associated with the class where the resources were declared.

NOTE Calling use Resource::Silo from a different module will create a separate container instance. You'll have to re-export (or otherwise provide access to) this function.

This is done on purpose so that multiple projects or modules can coexist within the same interpreter without interference.

silo->new will create a new instance of the same container class.

TESTING: LOCK AND OVERRIDES

It's usually a bad idea to access real-world resources in one's test suite, especially if it's e.g. a partner's endpoint.

Now the #1 rule when it comes to mocks is to avoid mocks and instead design the modules in such a way that they can be tested in isolation. This however may not always be easily achievable.

Thus, Resource::Silo provides a mechanism to substitute a subset of resources with mocks and forbid the instantiation of the rest, thereby guarding against unwanted side-effects.

The lock/unlock methods in Resource::Silo::Container, available via silo->ctl frontend, temporarily forbid instantiating new resources. The resources already in cache will still be OK though.

The override method allows to supply substitutes for resources or their initializers.

The derived flag in the resource definition may be used to indicate that a resource is safe to instantiate as long as its dependencies are either instantiated or mocked, e.g. a DBIx::Class schema is probably fine as long as the underlying database connection is taken care of.

Here is an example:

use Test::More;
use My::Project qw(silo);
silo->ctl->lock->override(
    dbh => DBI->connect( 'dbi:SQLite:database=:memory:', '', '', { RaiseError => 1 ),
);

silo->dbh;                   # a mocked database
silo->schema;                # a DBIx::Class schema reliant on the dbh
silo->endpoint( 'partner' ); # an exception as endpoint wasn't mocked

CAVEATS AND CONSIDERATIONS

See Resource::Silo::Container for resource container implementation.

CACHING

All resources are cached, the ones with arguments are cached together with the argument.

FORKING

If the process forks, resources such as database handles may become invalid or interfere with other processes' copies. As of current, if a change in the process ID is detected, the resource cache is reset altogether.

This may change in the future as some resources (e.g. configurations or endpoint URLs) are stateless and may be preserved.

CIRCULAR DEPENDENCIES

If a resource depends on other resources, those will be simply created upon request.

It is possible to make several resources depend on each other. Trying to initialize such resource will cause an expection, however.

COMPATIBILITY

As of current, using Resource::Silo -class and Moose in the same package doesn't work.

Usage together with Moo works, but only if Resource::Silo comes first:

package My::App;
use Resource::Silo -class;
use Moo;

has config_name => is => ro, default => sub { '/etc/myapp/myfile.yaml' };

resource config => sub { LoadFile( $_[0]->config_name ) };

Compatibility issues are being slowly worked on.

MORE EXAMPLES

Resources with just the init

package My::App;
use Resource::Silo;

resource config => sub {
    require YAML::XS;
    YAML::XS::LoadFile( "/etc/myapp.yaml" );
};

resource dbh    => sub {
    require DBI;
    my $self = shift;
    my $conf = $self->config->{database};
    DBI->connect(
        $conf->{dbi}, $conf->{username}, $conf->{password}, { RaiseError => 1 }
    );
};

resource user_agent => sub {
    require LWP::UserAgent;
    LWP::UserAgent->new();
    # set your custom UserAgent header or SSL certificate(s) here
};

Note that though lazy-loading the modules is not necessary, it may speed up loading support scripts.

Resources with extra options

resource logger =>
    cleanup_order   => 9e9,     # destroy as late as possible
    require         => [ 'Log::Any', 'Log::Any::Adapter' ],
    init            => sub {
        Log::Any::Adapter->set( 'Stderr' );
        # your rsyslog config could be here
        Log::Any->get_logger;
    };

resource schema =>
    derived         => 1,        # merely a frontend to dbi
    require         => 'My::App::Schema',
    init            => sub {
        my $self = shift;
        return My::App::Schema->connect( sub { $self->dbh } );
    };

Resource with parameter

An useless but short example:

#!/usr/bin/env perl

use strict;
use warnings;
use Resource::Silo;

resource fibonacci =>
    argument            => qr(\d+),
    init                => sub {
        my ($self, $name, $arg) = @_;
        $arg <= 1 ? $arg
            : $self->fibonacci($arg-1) + $self->fibonacci($arg-2);
    };

print silo->fibonacci(shift);

A more pragmatic one:

package My::App;
use Resource::Silo;

my %known_namespaces = (
    lock    => 1,
    session => 1,
    user    => 1,
);

resource redis  =>
    argument        => sub { $known_namespaces{ $_ } },
    require         => 'Redis::Namespace',
    init            => sub {
        my ($self, $name, $ns) = @_;
        Redis::Namespace->new(
            redis     => $self->redis,
            namespace => $ns,
        );
    };

resource redis_conn => sub {
    my $self = shift;
    require Redis;
    Redis->new( server => $self->config->{redis} );
};

# later in the code
silo->redis;            # nope!
silo->redis('session'); # get a prefixed namespace

Overriding in test files

use Test::More;
use My::App qw(silo);

silo->ctl->override( dbh => $temp_sqlite_connection );
silo->ctl->lock;

my $stuff = My::App::Stuff->new();
$stuff->frobnicate( ... );        # will only affect the sqlite instance

$stuff->ping_partner_api();       # oops! the user_agent resource wasn't
                                  # overridden, so there'll be an exception

Fetching a dedicated resource instance

use My::App qw(silo);
my $dbh = silo->ctl->fresh('dbh');

$dbh->begin_work;
# Perform a Big Scary Update here
# Any operations on $dbh won't interfere with normal usage
#     of silo->dbh by other application classes.

SEE ALSO

Bread::Board - a more mature IoC / DI framework.

ACKNOWLEDGEMENTS

  • This module was names after a building in the game Heroes of Might and Magic III.

  • This module was inspired in part by my work for Cloudbeds. That was a great time and I had great coworkers!

BUGS

Please report any bugs or feature requests to https://github.com/dallaylaen/resource-silo-p5/issues or via RT: https://rt.cpan.org/NoAuth/ReportBug.html?Queue=Resource-Silo.

SUPPORT

You can find documentation for this module with the perldoc command.

perldoc Resource::Silo

You can also look for information at:

COPYRIGHT AND LICENSE

Copyright (c) 2023, Konstantin Uvarin, <khedin@gmail.com>

This program is free software. You can redistribute it and/or modify it under the terms of either: the GNU General Public License as published by the Free Software Foundation, or the Artistic License.

See http://dev.perl.org/licenses/ for more information.