NAME

TipJar::MTA - outgoing SMTP with exponential random backoff.

SYNOPSIS

use TipJar::MTA '/var/spool/MTA';	# must be a writable -d
# defaults to ./MTAdir
$TipJar::MTA::interval='100';		# the default is 17
$TipJar::MTA::TimeStampFrequency='35';	# the default is 200
$TipJar::MTA::AgeBeforeDeferralReport=7000;	# default is 4 hours
$TipJar::MTA::MyDomain='peanut.af.mil';	# defaults to `hostname`
# bouces to certain matching addresses can be suppressed.
@TipJar::MTA::NoBounceRegexList = map { qr/$_/} (
  '^MDA-bounce-recipient\@tipjar.com$',
  '^RAPNAP\+challenge\+sent\+to\+.+==.+\@pay2send.com$'
);
# And away we go,
TipJar::MTA::run();			# logging to /var/spool/MTA/log/current

alternately,

use TipJar::MTA '/var/spool/MTA', 'nodns';  # we are sending to
# a restricted set of domains
# or using a smarthost
%TipJar::MTA::SMTProutes = (
 SMARTHOST => 'smtp_outbound.internal',  # smarthost for forwarding, can be a list too
 'example.com' => # mail to example.com will be randomly routed through these three
 [qw/  smtp1.example.com smtp2.example.com backup-smtp.example.org /],
 'example.net' => 'bad-dog.example.net'  # all mail to example.net goes to bad-dog
);

DESCRIPTION

On startup, we identify the base directory and make sure we can write to it, check for and create a few subdirectories, check if there is an MTA already running and stop if there is, so that TipJar::MTA can be restarted from cron.

We are not concerned with either listening on port 25 or with local delivery. This module implements outgoing SMTP with exponentially deferred random backoffs on temporary failure. Future delivery scheduling is determined by what directory a message appears in. File age, according to stat(), is used to determine repeated deferral.

We reuse a socket to a domain if we had trouble connecting to the MX for that domain in the past, or for multiple new messages going to the same domain. We also cache 4XX and 5XX error codes on recipients for four hours to eliminate a mess of traffic when, for instance, we have to bounce many messages to the same bogus return address. We will get a "550 User Unknown" error on the first bounce and throw away the others.

Every $interval seconds, we fork a child process.

A new child process first goes through all new outbound messages and expands them into individual messages and tries to send them. New messages are to be formatted with the return address on the first line, then recipient addresses on subsequent lines, then a blank line (rather, a line with no @ sign), then the body of the message. The TipJar::MTA::queue module will help compose such files if needed.

As of version 0.30, multiple recipients on the first line of a queued file will all be attempted, to the mx of the domain of the host in the first recipient.

Messages are rewritten into multiple messages when they are for multiple recipients, and then attempted in the order that the recipients appeared in the file.

After attempting new messages, a child process attempts all messages in the "immediate" directory.

After attempting all messages in the immediate directory, a child process moves deferred messages whose times have arrived into the immediate directory for processing by later children.

Deferred messages are stored in directories named according to when a message is to be reattempted. Reattempt times are assigned at requeueing time to be now plus between three and five quarters of the message age. Messages more than a week old are not reattempted. An undeliverable message that got the maximum deferrment after getting attempted just shy of the one-week deadline could conceivably be attempted for the final time fifteen and three quarters days after it was originally enqueued. Then it would be deleted.

An array of regular expressions can be specified, and if any of them match the sender of a bouncing message, the bouncing is suppressed, so you don't have to waste time with bounce messages from bad addresses you're sending challenges to for instance.

format for new messages

The format for new messages is as follows:

return address

The first line of the message contains the return address. It can be bare or contained in angle-brackets. If there are angle brackets, the part of the line not in them is discarded.

recipient list

All recipients are listed each on their own line. Recipients must have at-signs in them.

blank line

The first line (after the first line) that does not contain a @ symbol marks the end of the recipients. We are not concerned with local delivery.

data

Follow the routing information with the data, starting with header lines.

EXPORT

None.

DEPENDENCIES

the dnsmx() function uses Net::DNS. Versions 0.14 and previous use djbdns' dnsmx tool if that's preferable for you -- the old function is commented out.

The file system holding the queue must support reading from a file handle after the file has been unlinked from its directory. If your computer can't do this, see the spot in the code near the phrase "UNLINK ISSUE" and follow the instructions.

For that matter, we also generate some long file names with lots of dots in them, which could conceivably not be portable.

NODNS OPERATION

beginning with version 0.20, the dependency on Net::DNS can be skipped by including the term "nodns" on the use line, after the MTAdir, which must appear, to avoid changing the interface. When nodns is declared, all MX lookups will be directly from the %TipJar::MTA::SMTProutes hash of array references keyed by lowercased domain names. If the desired domain does not appear, the reserved domain name 'SMARTHOST' is looked up as a fallback or a list of fallbacks. If no SMARTHOST is declared, an error will be thrown.

The smtproute is selected from the listed routes randomly.

HISTORY

0.03 17 April 2003

threw away some inefficient archtecture ideas, such as per-domain queues for connection reuse, in order to have a working system ASAP. Testing kill-zero functionality in test script.

0.04 19 April 2003

logging to $basedir/log/current instead of stdout, unless $LogToStdout is true. $AgeBeforeDeferralReport variable to suppress deferral bounces when a message has been queued for less than an interval.

0.05 22 April 2003

slight code and documentation cleanup

0.06 6 May 2003

Testing, testing, testing! make test on TipJar::MTA::queue before making test on this module, and you will send me two e-mails. Now using Sys::Hostname instead of `hostname` and gracefully handling absolutely any combination of carriage-returns and line-feeds as valid line termination.

0.07 1 June 2003

Wrapped all reads and writes to the SMTP socket in eval blocks, and installed a ALRM signal handler, for better handling of time-out conditions. Also added a $TipJar::MTA::TimeStampFrequency variable which is how many iterations of the main fork-and-send loop to make before logging a timestamp. The default is 200.

0.08 10 June 2003

minor cleanup.

0.09 12 June 2003

AOL and Yahoo.com and who knows how many other sticklers require angle brackets around return addresses and recipients. Improved handling of MXes that we cannot connect to, by defining a ReQueue_unconnected entry point in addition to the ReQueue one that we had already..

0.10 20 June 2003

We now bounce mail to domains that ( have no MX records OR there is only one MX record and it is the same as the domain name ) AND we could not resolve the one name. Previously it had been given the full benefit of th doubt.

0.11 late June 2003

implemented domain listing for connection reuse. New messages and messages queued due to connection failure (but not 400 codes) get listed in the per-domain queue.

0.12 18 July 2003

fixed a bug that caused the earlier of multiple messages handled in the same batch to get clobbered. Re-engineering domain file locking too.

0.13 20 July 2003

we can now handle multi-line 250 responses. They exist. Also fixed a problem with retry deferral. And lowercasing of domain. And SMTP peers who give you a tiny packet on connection before they send their 220 greeting. And several "uninitialized value" warnings.

Adding a framework for remembering and caching recipient-based 4* and 5* errors for four hours.

0.14 23 July 2003

the path to the dnsmx program is now configurable through a global variable $TipJar::MTA::dnsmxpath

The dependency on the dateheader package is now in the Makefile.PL

we now remember domains we have had trouble connecting to and wait at least $TipJar::MTA::ConRetryDelay seconds (defaults to seventeen minutes) after a failed connection attempt before trying again.

We can handle peers that don't know what to do with a RSET command. It no longer shuts us down, rather we remember that the peer doesn't know how to reset its buffers and we prefer not to reuse a socket when sending them messages.

0.17 15 Feb 2004

rewrote dnsmx() to use Net::DNS

added @NoBounceRegexList configuration variable, which is a list of sender-matching regexes that we don't bounce to -- very useful when you're running a C/R system and have a lot of bogus addresses you don't care to hear about

We no longer get confused by multi-line greetings, which may have been the source of many earlier confusions

log text changes

0.18 30 Mar 2005

new conf variable $MaxActiveKids determines max parallel

fixed problem with zero-length message files

4xx error codes are now only cached $FourErrCacheLifetime seconds (defaults to 7 * 60 )

error code cache cleanup is fixed

0.20 21 May 2007

now support 'nodns' use-line option to suppress loading Net::DNS and SMTProutes hash to provide hard-coding of mail exchange paths instead. And revised the licence.

0.21 22 May 2007

moving newly appeared files from basedir into tempdir with unique names, instead of locking them, for all those situations where reading a file that was unlinked after it was opened just doesn't work.

0.30 31 Oct 2008

the call to sleep($interval) is now checked and retried when it wakes up early due to the CHLD handler. Also there is new code to handle multiple recipients per SMTP transaction, which are organized by preferred MX and split into smaller groups when they are delayed at RCPT TO time. Yes this made things more complex, and it is not clear that it is bugless, so expect a 0.31 fairly soon.

0.31 13 April 2009

Fixed a problem with requeueing messages with one recipient in 4XX situations. Added a new header to remember requeueing times

0.32 24 April 2009

Two patches from RT queue, one quite old.

Thanks to Tim Esselens, the long-standing bug about error codes not actually expiring after four hours like they are supposed to has been repaired

Thanks to Jose Luis Martinez, The address to choose for the local end of sockets is now configurable through a global variable $TipJar::MTA::BindAddress, also a situation that resulted in false bounce-as-undeliverables has been mitigated.

my ... if 0 style statics have been refactored into outer scopes, and other style points causing warnings under 5.10 have been addressed as well.

0.33 11 June 2009

Thanks again JLMARTINEZ for bug #45702 and fix for it

We no longer send spaces after colons for compliance with RFC 5321 section 3.3.

0.34 Spring 2010

we now have a TipJar::MTA::once subroutine that goes through the queue once then returns, also some modifications for working well on Windows

To-do list and Known Bugs

Patches are welcome.

log rolling

there is no rotation of the log in the mylog() function. mylog does reopen the file by name on every logging event, though. Rewriting mylog to use Unix::Syslog or Sys::Syslog would be cool, but would add dependencies. Mailing the log to the postmaster every once in a while is easy enough to do from cron.

ESMTP

take advantage of post-RFC-821 features, specifically PIPELINING

QMTP

use QMTP when available.

local deliveries

add MBOX and MailDir deliveries and some kind of configuration interface

LICENSE

see the README file for the license, which has changed. (you must submit your patches!)

AUTHOR

David Nicol, <davidnico@cpan.org>

SEE ALSO

TipJar::MTA::queue.

cpanm TipJar::MTA

perl -MCPAN -e shell
install TipJar::MTA