NAME

Doit - a scripting framework

SYNOPSIS

use Doit; # automatically does use strict + warnings
my $doit = Doit->init;
$doit->...;

Some boilerplate is needed if parts of the Doit script should be used on a remote machine or as another user — basically the script should be written like a "modulino":

use Doit;
sub function_which_may_run_on_remote {
    my($doit, @args) = @_;
    ...
    return ...;
}
return 1 if caller;
my $doit = Doit->init;
{
    my $ssh = $doit->do_ssh_connect('user@host'); # will sync all necessary bits to remote automatically
    my $result = $ssh->call_with_runner('function_which_may_run_on_remote', $arg, ...);
}
{
    my $sudo = $doit->do_sudo;
    my result = $sudo->call_with_runner('function_which_may_run_on_remote', $arg, ...);
}

Call a Doit-using script in dry-run mode:

doit-script.pl --dry-run [other parameters]

Call a Doit-using script in real mode:

doit-script.pl [other parameters]

DESCRIPTION

Doit is a scripting framework. Some core principles implemented here are:

Failing commands throw exceptions — similar to autodie or Fatal (but implemented consistently) or Bourne shell's set -e, or make(1)'s default mode without -k
Commands are checked first whether execution is required — making it possible to write "converging" scripts
Command execution is logged — like with Bourne shell's set -x or make's default mode
There's a dry-run mode which just shows what would happen — like make's -n switch

Doit scripts are normal Perl scripts which happen to run Doit commands — no limiting DSL involved, but the full expressiveness of Perl is available.

To achieve the principles it's required to wrap existing functions. A number of Perl builtins and module functions which do side-effects on a system (mostly changes on the file system) are available as Doit commands. Additionally there's a component system for supporting typical tasks like managing system packages, adding users, dealing with source repositories.

Additionally it's possible to run Doit functionality on remote servers (through ssh(1)) or as different users (using sudo(8)).

It is possible to create Doit scripts for running in bootstrapping situations. This means that prerequisites for Doit should be minimal. No mandatory CPAN modules are required, just standard Perl modules. Only for remote connections Net::OpenSSH is needed on the local side. For convenient system command execution IPC::Run may be used. Scripts run with Perl 5.8.x (maybe even 5.6.x is possible).

CONSTRUCTOR

my $doit = Doit->init;

Generates an object (technically it's a Doit::Runner object) which is used for calling Doit commands. The constructor looks for a command-line option --dry-run (or the short -n alias) and configures the runner for dry-run mode (just print what would be executed), otherwise for real mode (actually execute everything). Other command-line options are still available and may be used in the script, e.g. by using Getopt::Long or looking into @ARGV:

use Doit;
use Getopt::Long;
my $doit = Doit->init; # already handles --dry-run and -n
GetOptions(...)
    or die "usage: ...";
my @files = @ARGV
    or die "usage: ...";

CORE COMMANDS

All core commands throw exceptions on errors. If not stated otherwise, then the return value is the number of changes, typically the number of files affected --- in dry-run mode it's the number of changes which would be done, and in real mode it's the number of changes performed.

chmod

$doit->chmod($mode, $file ...);
$doit->chmod({quiet => $bool}, $mode, $file ...);

Make sure that the permission of the listed files is set to $mode (which is typically expressed as an octal number). Fails if not all files could be changed. If quiet is set to a true value, then no logging is done. See "chmod" in perlfunc for more details.

chown

$doit->chown($user, $group, $file ...);
$doit->chown({quiet => $bool}, $user, $group, $file ...);

Make sure that the owner (and group) of the listed files is set to the given values. The user and group may be specified as uid/gid or as username/groupname. A value of -1 or undef for $user and $group is interpreted to leave that value unchanged. This command is not useful on Windows systems. If quiet is set to a true value, then no logging is done. See "chown" in perlfunc for more details.

create_file_if_nonexisting

$doit->create_file_if_nonexisting($file ...);

Make sure that the listed files exist. Contrary to the Doit touch command and the system command touch(1) this does nothing if the file already exists.

copy

$doit->copy($from, $to);
$doit->copy({quiet => $bool}, $from, $to);

Make sure that the file $from is copied to $to unless there's already a file with same contents. Copying is done with File::Copy::copy. File attributes are not copied — this can be done using Doit::Util::copy_stat.

The logging includes a diff between both files, if the diff(1) utility is available. This can be turned off by specifying the quiet=>1 option.

ln_nsf

$doit->ln_nsf($oldfile, $newfile);

Make sure that $newfile is a symlink pointing to $oldfile, possibly replacing an existing symlink. Implemented by running the system's ln -nsf. See ln(1) for more details and "symlink" for an alternative.

make_path

$doit->make_path($directory ...);
$doit->make_path($directory ..., { key => val ... });

Make sure that the listed directories exist, together with any missing intermediate directories. Additional options may be specified as key-value pairs in a hashref, and will be passed to File::Path::make_path.

Note that it's possible to set the directory permissions with the mode option, but the make_path command does not check if an already existing directory has these permission bits set.

mkdir

$doit->mkdir($directory);
$doit->mkdir($directory, $mode);

Make sure that the given $directory exist. The $mode will be used only if creating a new directory and not effective if the directory already exists. See "mkdir" in perlfunc for more details. See "make_path" for a command which will also create mising intermediate directories.

move

$doit->move($from, $to);

Move file $from to $to. This command probably cannot be used in converging scripts without an accompanying condition. See File::Copy::move for details. For an alternative command see "rename".

Always returns 1 (unless there's an exception).

remove_tree

$doit->remove_tree($directory ...);
$doit->remove_tree($directory ..., { key => $val ... });

Make sure that the listed directories don't exist anymore, together with containing files and sub-directories. See File::Path::remove_tree for details.

rename

$doit->rename($from, $to);

Rename $from to $to. This command probably cannot be used in converging scripts without an accompanying condition. See "rename" in perlfunc for details. See "move" for a command which can move a file between different filesystems.

Always returns 1 (unless there's an exception).

rmdir

$doit->rmdir($directory);

Make sure that the given $directory is removed. Fails if this directory is not empty. See "rmdir" in perlfunc for details.

setenv

$doit->setenv($key, $val);

Make sure that %ENV contains a key $key set to $value.

symlink

$doit->symlink($oldfile, $newfile);

Make sure that $newfile is a symlink pointing to $oldfile. Contrary to "ln_nsf" it does not change an existing symlink. See "symlink" in perlfunc for more details.

touch

$doit->touch($file ...);

"Touches" the given files. Loosely modelled after the system command touch(1). Non-existent files are created as empty files, and for existent files the access and modification are updated. This command does not converge; for a converging command see "create_file_if_nonexisting".

Always returns the number of given files (unless there's an exception).

unlink

$doit->unlink($file ...);

Make sure that the given files are deleted. See "unlink" in perlfunc.

unsetenv

$doit->unsetenv($key);

Make sure that %ENV does not contain the key $key anymore.

utime

$doit->utime($atime, $mtime, $file ...);

Make sure that access time and modification time of the listed files is set to the given values. Undefined time values are replaced by current time. Fails if not all files could be changed. See "utime" in perlfunc for details.

FILE CREATION AND MODIFICATION

change_file

$doit->change_file({debug => $bool, check => $code}, $file, { change ... } ...);

Modify an existing $file using the set of change specifications. Return the number of changes made. Depending on the changes the command call can be converging or not. The following change specifications exist:

{ add_if_missing => $line }: Add the specified $line to the end of file if it is missing.
{ add_if_missing => $line, add_after => $rx }: Add the specified $line after a the last line mathing $rx. If no line matches, then an exception will be thrown.
{ add_if_missing => $line, add_after_first => $rx }: Add the specified $line after a the first line mathing $rx. If no line matches, then an exception will be thrown.
{ add_if_missing => $line, add_before => $rx }: Add the specified $line before a the first line mathing $rx. If no line matches, then an exception will be thrown.
{ add_if_missing => $line, add_before_last => $rx }: Add the specified $line before a the last line mathing $rx. If no line matches, then an exception will be thrown.
{ match => $rx_or_string, replace => $line }: Substitute all lines matching $rx_or_string with $line. $rx_or_string may be a regexp, or a string. In the latter case the complete line has to match.
{ match => $rx_or_string, delete => $bool }: All lines matching $rx_or_string will be deleted if delete is set to a true value.
{ match => $rx, action => $code }: For all lines matching $rx_or_string call the specified $code reference. In the code reference $_[0] can be used to manipulate the line.
{ unless_match ... }: TBD

The following options may be set:

debug => $bool: Turn on debugging if set to a true value.
check => $code: Do a final check on the non-committed file. The $code reference will take the filename as parameter. Just throw an exception if the changes should not be committed.

Implementation details: a temporary copy is created first, which is then changed using Tie::File. If everything went right and changes had to be done, then the changed file is renamed to the final destination.

Using Tie::File has some consequences: it does not know anything about encodings, and it uses $/ to determine line endings, which by default is set to do the right thing for text files. In future these details may be streamlined or even changed.

write_binary

$doit->write_binary($filename, $content);
$doit->write_binary({quiet=>$level, atomic=>$bool}, $filename, $content);

Make sure that the file $filename has the content $content. Return 1 if a change was done, otherwise 0. The given content should be unencoded (raw, binary, octets). If you need a specific encoding, then it has to be encoded before:

$doit->write_binary($filename, Encode::encode_utf8($character_content));

If quiet is set to 1, then no diffs are shown, otherwise a diff is shown if the file contents changed (and the diff(1) utility is available), or the complete contents are shown for a new file. If quiet is set to 2, then no logging at all is done.

By default, the file is written atomically by writing to a temporary file first, and then renamed. This can be changed by setting atomic=>0.

The command is modelled after File::Slurper::write_binary, but implemented without any dependencies.

Note: currently there's no write_text, but maybe will be added in future.

SYSTEM EXECUTION COMMANDS

A number of commands exist for executing system commands. All of these (except for cond_run) are non-converging and probably should be run conditionally.

All of the commands may be used with list syntax to avoid usage of a shell. This is also especially true on Windows, where perl's system has problematic edge cases.

A quick overview:

"run" - most comprehensive, but requires the non-core module IPC::Run
"qx" - may capture stdout
"open2" - may capture stdout and provide stdin
"open3" - may capture stdout and stderr and provide stdin
"system" - just run the command, cannot capture anything
"info_*" - variants which also run in dry-run mode, see "SYSTEM EXECUTION INFORMATIONAL COMMANDS"
"cond_*" - variants which are run conditionally

cond_run

$doit->cond_run(if      => sub { ... }, cmd => ["command", "arg" ...]);
$doit->cond_run(unless  => sub { ... }, cmd => ["command", "arg" ...]);
$doit->cond_run(creates => $file,       cmd => ["command", "arg" ...]);

Conditionally run the command specified in cmd (an array reference with command and arguments). Conditions are expressed as code references which should return a true or false value (options if for a positive condition and unless for a negative condition), or with the option creates for checking the existence of the given $file which is expected to be created by the given command. Conditions may be combined.

The cmd option may also specify a IPC::Run-compatible list, for example:

$doit->cond_run(creates => $file, cmd => [["command", "arg" ...], ">", $file]);

Return 1 if the condition was true and the cmd executed, otherwise 0.

open2

my $stdout = $doit->open2("command", "arg" ...);
my $stdout = $doit->open2({quiet => $bool, info => $bool, instr => $input}, "command", "arg" ...);

open3

my $stdout = $doit->open3("command", "arg" ...);
my $stdout = $doit->open3({quiet     => $bool,
                           info      => $bool,
                           instr     => $input,
                           errref    => \$stderr,
                           statusref => \%status, 
                          }, "command", "arg" ...);

Execute a command and return the produced stdout. If quiet is set to a true value, then no logging is done. If info is set to a true value, then command execution happens even in dry-run mode. If instr is specified, then it's send to the stdin of the command. If errref is set to a scalar reference, then this is filled with the stderr of the command; otherwise stderr won't show up. If statusref is set to a hash reference, then it is filled with the exit information of the command: msg, errno, exitcode, signalnum, coredump (except for msg fields may be missing).

Implementation is done with IPC::Open3.

qx

my $stdout = $doit->qx("command", "arg" ...);
my $stdout = $doit->qx({quiet => $bool, info => $bool, statusref => \%status}, "command", "arg" ...);

Implementation is done with Safe Pipe Opens.

run

$doit->run(...);

Execute a command using IPC::Run. The command specification may contain pipes, redirects and everything IPC::Run supports. Example:

$doit->run([qw(grep pattern file)], '|', ['sort'], '|', [qw(uniq -c)], '>', 'outfile');

Always returns 1 (unless there's an exception).

system

$doit->system("command", "arg" ...);
$doit->system({quiet => $bool, info => $bool, show_cwd => $bool}, "command", "arg" ...);

Execute a command. If show_cwd is set to a true value, then logging shows also the current working directory. If quiet is set to a true value, then no logging is done. If info is set to a true value, then command execution happens even in dry-run mode. See "system" in perlfunc for more details.

Always returns 1 (unless there's an exception).

SYSTEM EXECUTION INFORMATIONAL COMMANDS

Commands starting with info_ also run in dry-run mode. It is expected that the user only runs system commands which are not doing any changes to the system, but just return some kind of "information".

Note: currently the info_* commands fail on non-zero exit code. This behavior is probably not very useful (just think of running a non-matching grep) and may change in future. Currently these invocations have to be wrapped in an eval { ... } if non-zero exit may happen.

info_open2

my $stdout = $doit->info_open2("command", "arg" ...);
my $stdout = $doit->info_open2({quiet => $bool, instr => $input}, "command", "arg" ...);

Like "open2", but with the option info=>1 set.

info_open3

my $stdout = $doit->info_open3("command", "arg" ...);
my $stdout = $doit->info_open3({...}, "command", "arg" ...);

Like "open3", but with the option info=>1 set.

info_qx

my $stdout = $doit->info_qx("command", "arg" ...);
my $stdout = $doit->info_qx({quiet => $bool, statusref => \%status}, "command", "arg" ...);

Like "qx", but with the option info=>1 set.

info_system

$doit->info_system("command", "arg" ...);
$doit->info_system({quiet => $bool, show_cwd => $bool }, "command", "arg" ...);

Like "system", but with the option info=>1 set.

MISC INFORMATIONAL COMMANDS

Informational commands also run in dry-run mode.

which

$doit->which($program)

Return the path to the given $program, or undef if it cannot be found in user's PATH.

REMOTE AND USER-SWITCHING FUNCTIONS

It's possible to run Perl code or Doit functionality on remote servers, or as different users. Two commands exist to create a Doit runner-like object: "do_ssh_connect" for running code over a ssh connection, and "do_sudo" for running code as a different user using sudo.

This Doit runner-like object may execute all Doit commands. Additionally it's possible to call functions defined in the Doit script itself using "call_with_runner" and "call". The latter two may be preferable in some cases preferable, as every call involves some serialization and communication overhead, also the current serialization method (Storable) limits the possible parameter and result types (i.e. regexps cannot be transferred, which is needed for some Doit commands like "change_file").

For remote and sudo operation the active Doit script together with Doit.pm and required components are sent to the destination system. The Doit script is loaded using require there — which is the reason why the script has to be written like a modulino: the "main" part of the script must not re-run again.

Remote and sudo operation work best if things are setup to be password-less. Using a ssh agent helps here, and if you may, define NOPASSWD in the /etc/sudoers rules. It's still possible to run scripts which require manual password input, but some setups like combining ssh and sudo may be tricky.

dry-run mode, if defined, is passed to the other system or user.

do_ssh_connect

my $ssh = $doit->do_ssh_connect('user@host', options ...);

my $net_openssh = Net::OpenSSH->new('user@host', options ...);
my $ssh = $doit->do_ssh_connect($net_openssh, options ...);

Create a Doit runner-like object which runs commands over a ssh connection to user@host. The connection and communication is created and done using Net::OpenSSH. It is also possible to pass a Net::OpenSSH object instead of specifying user and host in the first argument.

The following options are available:

debug => $bool

Turn communication-level debugging on.

as => $username

Run as a different user on remote side. Switching user is done with sudo. If the switch is not password-less, then probably something like tty should be passed in the options.

forward_agent => $bool

Enable ssh agent forwardning if set to true.

tty => $bool

Allocate a pseudo terminal. May be useful if the script requires interactive input (e.g. password input or host key confirmation). Note that simultanously having multiple ssh connections with tty enabled is calling for trouble.

port => $port

Use a different ssh port.

master_opts => [ ... ]

Additional options to pass to the underlying Net::OpenSSH object. See "master_opts" in Net::OpenSSH. Cannot be used if a Net::OpenSSH object was passed in this method call.

For example, to specify a connection timeout:

$doit->do_ssh_connect('user@host', master_opts => [-o => "ConnectTimeout=2"]);

Or using another identity:

$doit->do_ssh_connect('user@host', master_opts => [-i => $bootstrap_privkey_file]);

put_to_remote => $method

Method to copy files to the remote side. Possible options are rsync_put (default) and scp_put.

perl => $path

Use a different perl for running the commands. Defaults to $^X.

umask => $umask

Specify an explicit umask for the worker process. Default is unset, which means that the system's default umask is used, which may be 0022 or 0077.

do_sudo

my $sudo = $doit->do_sudo(options);

Create a Doit runner-like object which runs in a different user context. By default it runs as root. The following options are available:

sudo_opts => [ ... ]

An array reference of options passed to the sudo command. For example, to run as another user than root, try:

..., sudo_opts => ['-u', $username], ...

debug => $bool

Turn communication-level debugging on.

perl => $path

Use a different perl for running the commands. Defaults to $^X.

call_with_runner

my $result = $ssh->call_with_runner('function', 'arg' ...);
my $result = $sudo->call_with_runner('function', 'arg' ...);

Call a function in a remote or switched user context. The function will get the Doit runner object as the first argument. Other arguments are serialized and sent to the function. The function's result is also serialized and sent back. Context is preserved.

Example:

sub print_hostname {
    my($doit) = @_;
    $doit->system('hostname'); # will print the remote hostname
}
...
return 1 if caller;
...
my $ssh = $doit->do_ssh_connect('user@host');
$ssh->call_with_runner('print_hostname');

call

my $result = $ssh->call('function', 'arg' ...);
my $result = $sudo->call('function', 'arg' ...);

Like "call_with_runner", but don't pass a Doit runner object.

COMPONENTS

Doit components are Perl modules which may define additional commands mixed into the Doit runner object. A component is added by calling the "add_component" method:

$doit->add_component('git');

The component name is written lowercase without the Doit:: prefix of the implementing module, that is, for Doit::Git one uses the name git.

The component commands are typically prefixed with the component name. For example, the fbsdpkg component defined the fbsdpkg_install_packages and fbsdpkg_missing_packages commands.

Components added with add_component are also synced to remote systems for subsequent connections.

The following components are available:

System Packages

Doit::Brew - handle homebrew packages for Mac OS X
Doit::Deb - handle Debian packages
Doit::Fbsdpkg - handle FreeBSD packages
Doit::Gem - handle Ruby gem packages
Doit::Rpm - handle RPM packages

Security

Doit::Macsecurity - handle certificate management on Mac OS X
Doit::Ssl - handle certificate management for OpenSSL systems

Other

Doit::File - additional file commands
Doit::Git - handle git repositories
Doit::Locale - handle locale installation
Doit::Lwp - WWW access
Doit::User - user management

ENVIRONMENT

Automatically set

DOIT_IN_REMOTE

Set to a true value if a Doit-using script is currently run "remotely". Useful to do things in a Doit script only on the local system, for example:

use if !$ENV{DOIT_IN_REMOTE}, lib => "$ENV{HOME}/src/Doit/lib";
use Doit;
...

DOIT_SCRIPT

Set to the (potentially relative) path of the Doit-using script for further runners (ssh, sudo).

PWD

Doit::Util::in_directory keeps track of the standard PWD variable.

User-settable variables

DOIT_TRACE

May be set to a true value to enable command and connection tracing.

DOIT_TRACE=1 some_doit_script.pl

TERM

If set on a non-Windows system (which is usually the case in a normal Unix terminal session), then coloring in Doit::Log is enabled.

AUTHOR

Slaven Rezic <srezic@cpan.org>

	Global
`s`	Focus search bar
`?`	Bring up this help dialog

	GitHub
`g` `p`	Go to pull requests
`g` `i`	go to github issues (only if github is preferred repository)

	POD
`g` `a`	Go to author
`g` `c`	Go to changes
`g` `i`	Go to issues
`g` `d`	Go to dist
`g` `r`	Go to repository/SCM
`g` `s`	Go to source
`g` `b`	Go to file browse

	Search terms
module: (e.g. module:Plugin)
distribution: (e.g. distribution:Dancer auth)
author: (e.g. author:SONGMU Redis)
version: (e.g. version:1.00)

NAME

SYNOPSIS

DESCRIPTION

CONSTRUCTOR

CORE COMMANDS

chmod

chown

create_file_if_nonexisting

copy

ln_nsf

make_path

mkdir

move

remove_tree

rename

rmdir

setenv

symlink

touch

unlink

unsetenv

utime

FILE CREATION AND MODIFICATION

change_file

write_binary

SYSTEM EXECUTION COMMANDS

cond_run

open2

open3

qx

run

system

SYSTEM EXECUTION INFORMATIONAL COMMANDS

info_open2

info_open3

info_qx

info_system

MISC INFORMATIONAL COMMANDS

which

REMOTE AND USER-SWITCHING FUNCTIONS

do_ssh_connect

do_sudo

call_with_runner

call

COMPONENTS

ENVIRONMENT

Automatically set

User-settable variables

AUTHOR

COPYRIGHT

SEE ALSO

Module Install Instructions