Revision history for Perl extension Helios.
2.72_0950 Fri Feb 28 16:36:48 2014
HELIOS 2.80 RELEASE CANDIDATE 1
-Resolved service/service_name inconsistency in Helios::Config
API. Accessor method is set/getService, and the
get/set/unsetParam methods take 'service' as a parameter. The
set/getServiceName() methods are retained for backward
compatibility.
-Changed WORKER_LAUNCH_PATTERN names to be more descriptive:
linear, dynmaic, optimistic. As before, linear is the default.
-Filled gaps in Helios::Configuration guide and helios.pl POD.
-Updated copyright info.
2.71_4770 Sun Nov 24 21:39:28 2013
- Removed .pl extension from all new commands.
- Helios::Config->setParam(): fix so false values can be set.
- helios.pl: tweaks to database reconnect code.
- helios.pl: changes to command line option parsing so multiple
jobtypes can be specified with multiple --jobtypes options.
- helios.pl: removed all use Error-based try {} catch {} blocks.
This will allow future Helios versions to use exceptions based
on Exception::Class rather than Error.
- Helios::TheSchwartz has been removed; Helios::TS has completely
replaced it.
2.71_4460 Sat Nov 2 13:36:52 2013
Minor changes and fixes to new commands:
- helios_config_get.pl: fixed problem with not returning param
values set in helios.ini.
- helios_config_set.pl: fixed problem with setting param values
that evaluate to false.
- helios_config_unset.pl: changed output to make it more
consistent with the other new commands.
- helios_job_status.pl, helios_job_info.pl: specifying --jobid
no longer required; the first arg is assumed to be the jobid.
- helios_job_submit.pl: -v and -n now work in addition
to --verbose and --no-validate.
- helios_service_status.pl: separated service version to its own
line in output. Added '--epoch-time' option to report time in
epoch seconds instead of localtime() format.
2.71_4350 Fri Oct 25 23:23:29 2013
- NEW helios_service_status.pl - report service status.
- NEW helios_job_status.pl - report job exitstatus (if completed).
- NEW helios_job_info.pl - report job metainfo, args, and log msgs.
- helios_config_*.pl: added documentation.
- Helios::Config - fixed warning using getParam().
2.71_4250 Fri Oct 18 16:21:46 2013
- Helios::Config:
- NEW set/get/unsetParam() methods to more easily work
with individual config parameters in the HELIOS_PARAMS_TB
table.
- NEW helios_config_*.pl commands to make Helios config params
more accessible to shell scripts and other utilities.
- parseConfDb(): combined the 2 separate database queries
into 1 to reduce database calls when config is updated.
- separated Helios config parameter documentation into
Helios::Configuration POD.
- Helios::Service:
- Added JobLockInterval() functionality to control the
amount of time jobs are locked from a Helios API. Previously,
the only way to control the amount of time jobs were locked
was to override TheSchwartz::Worker's grab_for() method.
- Changed hostname lookup handling to reduce the number of calls
to Sys::Hostname::hostname().
- helios.pl:
- Changed cmd line option handling so --help and --version work
again.
- Added startup message to report the dsn of the collective
database helios.pl is connected to.
- Replaced most Error module-based try {} blocks with eval {}
- Updated POD with new cmd options and referenced
Helios::Configuration.
- Helios::Job - replaced usage of Helios::TheSchwartz with the
new Helios::TS class.
- Helios::JobType - added debug() accessor method.
- Makefile.PL: Added all helios_config_*.pl commands above and
helios_jobtype_add.pl to EXE_FILES. Removed old LICENSE line.
2.71_4051 Fri Oct 4 23:24:16 2013
- Added code for "virtual jobtypes" - jobtypes w/o a service
class. Allows a service to handle jobs of multiple jobtypes
without creating new service classes.
- NEW Helios::ObjectDriver class to improve driver init
handling.
- NEW Helios::JobType class to represent jobtypes; there should
no longer be a need to use TheSchwartz::FuncMap directly.
- NEW Helios::TS and Helios::TS::Job to:
- replace Helios::TheSchwartz as a full-featured TheSchwartz
subclass.
- implement virtual jobtypes.
- fix [RT79690] by dumping Helios::TS::Job objects w/o arg()
values before they are passed to the higher Helios layers.
- NEW helios_jobtype_add.pl to handle adding new jobtypes from
the command line.
- helios.pl:
- now handles @ARGV completely differently to support virtual
jobtype specification and add other options in future.
- uses Helios::TS instead of Helios::TheSchwartz
- Helios::Service:
- new, better constructor
- ton of NEW methods to support virtual jobtypes:
set/getJobType()
addAltJobType()
set/getAltJobTypes()
set/getAltJobtypeids()
addAltJobtypeid()
lookupAltJobtypeids()
lookupJobtypeid()
- jobsWaiting(): rewritten to support multiple, "virtual"
jobtypes
2.71_3860 Sat Sep 21 19:27:36 2013
- Added PRIORITIZE_JOBS feature.
- Added WORKER_LAUNCH_PATTERN feature.
2.61 Fri Sep 13 20:02:14 2013
- Helios 2.61 is a bugfix release:
- helios.pl: Fix for [RT81709].
- helios.pl: Fix for [RT81914].
- Helios::Service & Helios::Job: Patch for [RT79690]. This bug
actually appears to be caused by LOB binding problems with the
underlying DBD module. This problem can cause a loss of jobs
in the job queue under certain circumstances, so if you are
experiencing this problem you are encouraged to upgrade to this
version and upgrade your DBD module to the latest version.
Consult the rt.cpan.org page for this bug for more information.
- Makefile.PL: Changes for proper version handling of LICENSE.
Additions so the generated META.* files will have website,
bugtracking, and GitHub information for sites like MetaCPAN and
search.cpan.org.
2.60 Mon Nov 06 00:00:00 2012
-Significant changes to Helios database connection handling,
including a specialized Data::ObjectDriver class and TheSchwartz
extensions. Helios now aggressively caches database connections
at the DBI layer and uses DBI features to properly handle
database connect/disconnect between daemon and worker processes.
These improvements result in significant performance increases,
especially when using Oracle as a collective database. One
user's application experienced job throughput increases of ~70%.
MySQL collectives should also experience speed increases and
a reduced number of open connections.
-NEW Helios::Config modular configuration API. Similar to
Helios::Job and Helios::Logger, Helios::Config defines the basic
Helios configuration API while also serving as a base class to
build custom configuration modules for Helios applications. To
define a custom configuration API for a Helios application,
specify a ConfigClass() method in your Helios::Service subclass.
Auxiliary application pieces such as scripts and cronjobs can use
Helios::Config to get the application's configuration without
needing to instantiate the application class and calling the
prep() method as in the past. Helios::Config makes the Helios
configuration API more robust, modular, and flexible. See the
Helios::Config POD for more details.
NOTE: Helios::Service->getConfigFrom*() methods are now
deprecated in favor of Helios::Config->parseConfig(). The prep()
method uses Helios::Config and will continue to be supported.
-NEW Helios exception class for configuration errors:
Helios::Error::ConfigError.
-NEW WORKER_MAX_TTL_WAIT_INTERVAL parameter to fine-tune use of
WORKER_MAX_TTL with specific applications.
-NEW WORKER_BLITZ_FACTOR configuration parameter to launch
more workers sooner.
-NEW configuration parameter documentation. All Helios
out-of-box configuration parameters are now defined in a section
of the Helios::Config POD. For more information about the new
configuration parameters or any others, check the Helios::Config
POD!
-Helios::Service->dbConnect() has been updated to work with the
new database connection caching scheme and to better support
DBI options.
-Bundle::Helios::Generic has been changed to Bundle::Helios and
can be used to install Helios prerequisite modules with generic
Perl installs. Bundle::Helios::RedHat was dropped in 2.41. See
the INSTALL.* files in the main distribution directory to
determine the right type of install method for your environment.
-Improved module loading code.
-Removal of most instances of Error module's try {} catch {}
blocks in favor of Perl generic eval {} or do {}.
-More tests.
-Updated INSTALL.* files for generic Perl and CPAN installs and
Red Hat/CentOS 6.x. BSD, Mac OS X, and other *nix should use the
generic install instructions. (Debian and Ubuntu based
instructions continue to be the quickest and easiest. Thank you
Debian for your excellent Perl and CPAN packaging!)
2.41 Sun Jun 03 22:00:00 2012
-Official Oracle support! DDL for a Helios Oracle schema is in
'sql/helios_schema_oracle.sql'. Please note:
1) you will need to adjust the Oracle schema to fit your
environment (especially in regards to security and tablespaces)
2) while the Helios core supports Oracle, the Helios::Panoptes
web admin interface still only supports MySQL (Panoptes
support for Oracle is being worked on)
-Helios::Service and Helios::Job: added deferredJob() and
deferred() methods to defer processing of a job even though it
was available in the job queue. These methods (for now) simply
expose the declined() feature added in TheSchwartz 1.10; no Helios
job history is recorded for a deferred job. Existing Helios
users will need to upgrade their TheSchwartz library to 1.10 to
use this feature.
-Helios::Logger & Helios::Logger::Internal: moved driver
creation to base Helios::Logger class so (if necessary) any
loggers can get a Data::ObjectDriver to the collective database
-Helios::Service & Helios::Logger: refactored getDriver() into
initDriver() and getDriver() to eventually allow caching of
Data::ObjectDriver objects. Anything needing an ObjectDriver to
the collective database should just continue to call getDriver()
and everything will work fine.
-helios.pl: calls the service class prep() method on startup
rather than getConfigFromIni() and getConfigFromDb() separately.
This ensures proper setup of the loggers and Data::ObjectDriver
db connection.
-helios.pl: changed startup message so perl doesn't carp a warning
if the service class doesn't have a $VERSION defined.
-helios.pl: added registration_interval as a configuration
parameter (previously it was hard-coded to 300 seconds in
helios.pl). The "registration_interval" parameter is the amount
of time (in secs) between service daemon "collective check-ins"
(when a service daemon records its current status in the
HELIOS_WORKER_REGISTRY_TB table). This table is used to create
the Helios::Panoptes Collective Admin page, and is also useful
for system monitoring and statistics.
-helios.pl: set new defaults for zero_sleep_interval (10) and
registration_interval (60). These new defaults should make
Helios more responsive in low load situations and generally work
better for more users out-of-the-box. (The old defaults geared
Helios more for large volumes of long-running jobs on heavily
loaded databases. The new defaults should work for a better
range of small- to mid-sized loads.) If these new defaults
create too much traffic for your database, you can set new
larger values in your helios.ini file to decrease database
traffic.
-helios.pl: changes to prevent a worker from transmuting to a new
service daemon (caused by database instability during transition
between helios.pl, TheSchwartz->work(), and Helios::Service->work()
-updated tests
-submitJob.pl: fixed example in POD
-Removed Bundle::Helios::RedHat. Ultimately, it didn't work any
better (or worse) than using 'install Helios' command with the
CPAN shell. Bundle::Helios::Generic seems to work well on other
platforms, however. If you wish to use Helios with RHEL/CentOS
5.x or earlier, it is recommended you install a more recent
version of Perl than 5.8.8 for ease of installation reasons.
2.40 Sun Jan 22 22:30:00 2012
-NEW Extensible logging subsystem (Helios::Logger)
-NEW Helios::Error::BaseError exception class; acts as parent for
entire Helios::Error:: hierarchy
-NEW Helios::Service->metarun() method to modularize metajob
handling and provide for extensible metajob support
-NEW service run() and metarun() methods now run in an eval
block as a process safety measure to catch uncaught exceptions.
-NEW MP3IndexerService example application including in eg/
directory (StubService also rewritten for clarity)
-REVISED Helios::Service->work() method no longer pays attention
to return value of run() method (this feature was under-
documented, misunderstood, and caused problems with eval {} and
Try::Tiny exception handling
-REWRITTEN submitJob.pl CGI script; supports sending XML stream
in addition to form encoding, no longer dependent on
HELIOS_CLASS_MAP
-REWRITTEN Helios::TestService; more modern Perl and easier to
understand
-REWRITTEN install instructions, including specific instructions
for Red Hat- and Ubuntu-based systems, and including CPAN install
bundles (still experimental)
-REWRITTEN Helios::Tutorial
-Helios::Service: fixed MaxRetries() and RetryInterval() retry
job methods
-Helios::Job: changed failed() and failedNoRetry() to limit the
char length of the error recorded to 254 chars. That's one less
than the default size of the db field in the ERROR table, so it
should prevent long errors like stack traces from unexpectedly
blowing up worker processes and potentially losing jobs.
-Revised dependencies:
DBI 1.52 (shipped with RHEL/CentOS 5)
XML::Simple 2.14 (shipped with RHEL/CentOS 5)
Removed XML::SAX dependency (unnecessary)
Removed XML::Namespace dependency (unnecessary)
Removed XML::Parser dependency (unnecessary)
-Removed deprecated classes, methods, and other files:
Helios::Service->parseArgXML() (deprecated since 2.00)
Helios::Worker (deprecated since 2.00)
sql/helios_schema_20upgrade.sql
-Updated copyright notices
-Switch to "Linux-style" versioning:
odd minor version == development version
even minor version == stable version
second minor number == patch level
(we're still also using Perl-style _ for devel versions too)
2.23 Fri Sep 10 12:22:00 2010
-Helios::Job: fixed bug where getArgXML() returned undef if job
object was vivified from job queue
2.22 Thu May 21 11:30:00 2009
-Helios::Service: fixed a bug in getFuncidFromDb() that caused
services to fail to load properly if there was no entry for them
in FUNCMAP table.
-helios.pl: Several revisions to make it a more robust daemon. Now
honors TERM signal; 'kill <pid>' from a shell will attempt to
safely shut down workers and daemon process w/o using Panoptes
-Documentation updates
2.21 Thu May 14 17:00:00 2009
-added set/getJob() accessors to facilitate access to current
Helios::Job object from methods other than run()
2.20 Tue Apr 28 18:00:00 2009
-various tweaks for cross-platform compatibility
-documentation additions/corrections
-added Helios::LogEntry::Levels to remove dependencies on
Sys::Syslog
-helios.pl: Cleaned up messages to console after daemonize
-helios.pl + Helios::Service: added support for WORKER_MAX_TTL
config param
-Helios::Service: added JobClass() allow extensible Helios::Job
classes
-Helios::Service: added MaxRetries() and RetryInterval() to replace
TheSchwartz's max_retries() and retry_interval() (this is not an
abitrary change; it's to allow for extensions at the Helios layer
in the future)
-Helios::Service: added default run() method for class completeness
(it throws a Helios::Error::FatalNoRetry error because you forgot
to implement a run() method)
-Helios::Service: removed logging to syslogd to eliminate
Sys::Syslog dependency (see HeliosX::ExtLoggerService)
-Helios::Service: added set/getFuncid(), getFuncidFromDb() and
changed jobWaiting() to use them. This eliminates an unnecessary
database query previously done every time jobsWaiting() was called
-Helios::Service: added support for log_priority_threshold config
param
-Helios::Job: added getFailures() to access the number of times a
job has previously failed
2.00 Fri Aug 01 02:00:00 2008
-First public production release (same as 1.90_26 w/more documentation)
1.90_26 Thu Jun 26 11:30:00 2008
-Helios 2.0 RC1
-NEW Helios 2.0 API
Helios::Service
Helios::Job
Helios::MetajobBurstService
Helios::ConfigParam
Helios::JobHistory
Helios::LogEntry
Helios::TestService
-helios.pl: added PID check to prevent 2 instances of a service
running on the same server
-helios.pl: added --clear-halt option to clear Worker Admin HALTs
at the cmd line
-helios.sql: new helios_worker_registry_tb.start_time field for
uptime stats
-submitJob.pl, helios_job_submit.pl, helios_job_clean.pl use new
2.0 API
-eg/Stub/StubService.pm for example of new 2.0 API service class
-new tests to test both 1.x and 2.x API
1.19_07 Thu May 29 14:00:00 2008
-added Helios.pm (for version checking & documentation)
-added Helios::Job (for eventual refactoring)
-added Helios::Test (for system testing)
1.19_06 Wed May 28 12:00:00 2008
-added LICENSE file that was left out
-add helios_log_clean.pl utility to clean log, job history
-added eg/Stub/StubWorker.pm as a worker class stub example
1.19 Mon May 19 12:00:00 2008
-First public release