NAME
File::Tail::Dir - Tail all matching files in a given set of directories
SYNOPSIS
Typical usage:
use File::Tail::Dir;
my $tailer = File::Tail::Dir->new( filter => qr/.*access.*log$/,
processor => sub {
my ($filename, $lines) = @_;
...
},
);
$tailer->watch_files();Or, subclass:
package My::Tailer;
use Moose;
extends 'File::Tail::Dir';
# override standard 'process' method
sub process {
my ($self, $filename, $lines) = @_;
...
}DESCRIPTION
Monitors and processes any lines appended to the end of one or more files in a given list of directories, keeping state between sessions, and using kernel notification of file change events to maximise efficiency.
A list of directories is given to monitor, and filtering/exclude expressions allow just the files of interest to be selected for notification when lines are appended.
This module was originally created to support lossless logging of many Apache web server log files to a Scribe-based logging subsystem (see File::Tail::Scribe, Log::Dispatch::Scribe), and hence key requirements are to keep state, to be able to resume from the last known state between interruptions (like server reboots), and to follow the renaming and creation of new files during log rotation.
ALTERNATIVES
Like File::Tail::Multi, File::Tail::App, File::Tail::FAM, File::Tail, File::Tail::Dir is yet another module to tail and process one or more files. What File::Tail::Dir provides that is different is:
Tails all matching files within a given set of directories
Apart from File::Tail::Multi, the other alternatives allow monitoring of a single file only. File::Tail::Multi offers files and directories with glob pattern expansion and filtering. File::Tail::Dir offers a list of directories for monitoring with regular expression filtering and exclusions based on a list of strings and/or regular expressions.
Remembers state
File::Tail::Dir maintains a state file, so can be stopped and started without missing changes to files. As it is intended to work with log files that are periodically rotated, it will handle the situation where a file changes name between runs.
Uses kernel notification (where available) to monitor file changes
Built on Dave Rolsky's File::ChangeNotify, File::Tail::Dir supports all file notification methods offered by that module. For best performance on Linux, it is recommended that Linux::Inotify2 be installed (for kernels 2.6.13 and later). Of the similar modules listed above, only File::Tail::FAM provides event-based notification, using the older FAM kernel interface.
CONSTRUCTOR
new
$tailer = File::Tail::Dir->new(%options);Options are as follows:
- File Selection Options
-
directories => \@paths
Spefify the directory paths that should be watched for changes.
filter => qr/.../
This is an optional regular expression that will be used to check if a file is of interest. This filter is only applied to files. By default, all files are included.
exclude => [...]
An optional list of paths to exclude. This list can contain either plain strings or regular expressions. If you provide a string it should contain the full filesystem path to be excluded.
The paths can be either directories or specific files. If the exclusion matches a directory, all of its files and subdirectories are ignored.
follow_symlinks => $bool
By default, symbolic links in the filesystem are ignored. Set this to true to follow them.
If symlinks are being followed, symlinks to files and directories will be followed. Directories will be watched, and changes for directories and files reported.
(Note that if this is set to true, the presence of circular link references may cause problems due to a behaviour in the underlying File::ChangeNotify).
- Processing Options
-
processor => sub { my ($filename, $lines) = @_; ... }
Specifies the callback function to process the arrayref of lines appended to the given file.
The default action is to print the lines to standard output.
- Miscellaneous Options
-
sleep_interval => $number
Where kernel file change event notification is not available, this argument controls the interval between checks for file changes. The value is a number in seconds. The default is 2 seconds.
statefilename => $filename
Sets the name of the file used to store state between sessions. Defaults to '.filetaildirstate' in the working directory.
autostate => 1|0
Set to 1 to automatically store the state file periodically. Defaults to 0.
autostate_delay => $number
If autostate is enabled, sets the minimum delay in seconds between when the state file is automatically saved. If set to 0, state is saved on every file change.
no_init => 1|0
If no_init is set, the initial contents of the monitored files is skipped and only the changed contents (since creating the new File::Tail::Dir instance) is sent.
The default behaviour (no_init = 0) is for lossless logging, i.e. to send any contents that has been appended to existing files since last run, and to see all files without existing state as new files and therefore to send the whole contents of those files before monitoring for new changes.
max_age => $age_in_seconds
Maximum time in seconds to keep a filehandle open if no changes on the filehandle have been seen. Defaults to 3600 sec.
max_lines => $number
Maximum number of lines to process from a given file at a time. This sets a limit on the maximum amount of memory to use, particularly in the case of processing large files for the first time. Defaults to 10000.
watchdog_signal_name
Name of signal to use for watchdog. Defaults to USR1 on most systems, INT on Windows.
METHODS
watch_files
$tailer->watch_files()Blocking call to watch for changes in files forever. When a change is detected, the process() method is called.
process
This is the method invoked by watch_files() on file change, which in turn invokes the callback function specified by the 'processor' option. If no callback has been provided, it will print the filename and changed contents to standard output by default. This method may be overridden in a subclass instead of using the processor option. Its signature is:
sub process {
my ($self, $filename, $lines) = @_;
...
}The return value is ignored.
running
$running = $tailer->running(); # get status
$tailer->running(0); # exit event loopThis is set to true when the event loop in watch_files() is entered. It can be set to false (e.g. via a signal or in the process() method) to cause watch_files() to return after processing the current batch of events.
Note that if running is set to false via signal, the event loop will not be exited until a new file change event is seen.
save_state
$tailer->save_state();Saves internal state to file. This is always called when watch_files() completes normally, and is called periodically if autostate is enabled. However, you may wish to call it directly if you have installed a signal handler that might not allow watch_files() to complete.
__PACKAGE__->install_signal_handlers
File::Tail::Dir->install_signal_handlers();Installs default signal handlers. Once installed, on receiving any of HUP, INT, QUIT or TERM, all instances of File::Tail::Dir will attempt to save their state, and then exit.
If used, this should be called before any File::Tail::Dir instance is created.
AUTHOR
Jon Schutz, <jon at jschutz.net> http://notes.jschutz.net
BUGS
Please report any bugs or feature requests to bug-file-tail-dir at rt.cpan.org, or through the web interface at http://rt.cpan.org/NoAuth/ReportBug.html?Queue=File-Change-Dir. I will be notified, and then you'll automatically be notified of progress on your bug as I make changes.
SUPPORT
You can find documentation for this module with the perldoc command.
perldoc File::Tail::DirYou can also look for information at:
AnnoCPAN: Annotated CPAN documentation
CPAN Ratings
RT: CPAN's request tracker
Search CPAN
COPYRIGHT & LICENSE
Copyright 2012 Jon Schutz, all rights reserved.
This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.