NAME

check_jmx4perl - Nagios plugin using jmx4perl for accessing JMX data remotely

SYNOPSIS

# Check for used heap memory (absolute values)
check_jmx4perl --url http://localhost:8888/jolokia \
               --name memory_used \
               --mbean java.lang:type=Memory \
               --attribute HeapMemoryUsage \ 
               --path used \
               --critical 10000000 \
               --warning   5000000 

# Check that used heap memory is less than 80% of the available memory
check_jmx4perl --url http://localhost:8888/jolokia \
               --alias MEMORY_HEAP_USED \
               --base MEMORY_HEAP_MAX \ 
               --critical :80

# Use predefined checks in a configuration file with a server alias
# Server alias is 'webshop', check is about requests per minute for the 
# servlet 'socks_shop'
check_jmx4perl --config /etc/nagios/check_jmx4perl/tomcat.cfg
               --server webshop \
               --check tc_servlet_requests \
               --critical 1000 \
               socks_shop

# Check for string values by comparing them literally
check_jmx4perl --url http://localhost::8888/jolokia \
               --mbean myDomain:name=myMBean \
               --attribute stringAttribute \
               --string \
               --critical 'Stopped' \
               --warning '!Started'


# Check that no more than 5 threads are started in a minute
check_jmx4perl --url http://localhost:8888/jolokia \
               --alias THREAD_COUNT_STARTED \
               --delta 60 \
               --critical 5

# Execute a JMX operation on an MBean and use the return value for threshold
# Here a thread-deadlock is detected.
check_jmx4perl --url http://localhost:8888/jolokia \
               --mbean java.lang:type=Threading \
               --operation findDeadlockedThreads \
               --null no-deadlock \
               --string 1 \
               --critical !no-deadlock \
               --critical 10

# Use check_jmx4perl in proxy mode
check_jmx4perl --url http://localhost:8888/jolokia \
               --alias MEMORY_HEAP_USED \
               --critical 10000000 \
                --target service:jmx:rmi:///jndi/rmi://bhut:9999/jmxrmi

DESCRIPTION

check_jmx4perl is a Nagios plugin for monitoring Java applications. It uses an agent based approach for accessing JMX exposed information remotely.

Before start using check_jmx4perl an agent must be installed on the target platform. For JEE application server this is a simple webapplication packaged as a war archive. For other platforms, other agents are available, too. Please refer to the README for installation instructions and the supported platforms.

check_jmx4perls can also be used in an agentless mode (i.e. no agent needs to be installed on the target platform). See "Proxy mode" for details.

This plugin can be configured in two ways: Either, all required parameters for identifying the JMX information can be given via the command line. Or, a configuration file can be used to define one or more Nagios checks. This is the recommended way, since it allows for more advanced features not available when using the command line alone. Each command line argument has an equivalent option in the configuration files, though.

This documentation contains four parts. First, a tutorial gives a 5 minute quickstart for installing and using check_jmx4perl. The middle part offers some technical background information on JMX itself, the features provided by this plugin and finally the command line arguments and the configuration file directives are described.

TUTORIAL

Before we dive into the more nifty details, this 5 minutes quickstart gives a simple cooking recipe for configuration and setup of check_jmx4perl.

This tutorial uses tomcat as an application server. Download it from http://tomcat.apache.org (either version 5 or 6) and extract it:
```
$ tar zxvf apache-tomcat-*.tar.gz
$ # We need this variable later on:
$ TC=`pwd`/apache-tomcat*
```
Download jmx4perl from http://search.cpan.org/~roland/jmx4perl and install it:
```
$ tar zxvf jmx4perl-*.tar.gz
$ cd jmx4perl*
$ # Store current directory for later reference:
$ J4P=`pwd`      
$ perl Build.PL
$ sudo ./Build install
```
This is installs the Perl modules around JMX::Jmx4Perl which can be used for programmatic JMX access. There are some CPAN dependencies for jmx4perl, the build will fail if there are missing modules. Please install the missing modules via cpan (cpan module). The Nagios plugin check_jmx4perl is installed in a standard location (/usr/bin, /usr/local/bin or whatever your Perl installation thinks is appropriate) as well as the other scripts jmx4perl (a generic tool for accessing JMX) and j4psh (an interactive JMX shell).
Deploy the Jolokia agent in Tomcat:
```
$ cd $TC/webapps
$ jolokia
```
Start Tomcat:
```
$ $TC/bin/startup.sh
```
Check your setup:
```
$ jmx4perl http://localhost:8080/jolokia
```
This prints out a summary about your application server. http://localhost:8080/jolokia is the URL under which the agent is reachable. Tomcat itself listens on port 8080 by default, and any autodeployed war archive can be reached under its filename without the .war suffix (jolokia in this case).
Try a first Nagios check for checking the amount of available heap memory in relation to the maximal available heap:
```
$ check_jmx4perl --url http://localhost:8080/jolokia  \
                 --mbean java.lang:type=Memory    \
                 --attribute HeapMemoryUsage      \
                 --path used                      \
                 --base java.lang:type=Memory/HeapMemoryUsage/max \
                 --warning 80                     \
                 --critical 90        

OK - [java.lang:type=Memory,HeapMemoryUsage,used] : In range 9.83% (12778136 / 129957888) | 
    '[java.lang:type#Memory,HeapMemoryUsage,used]'=12778136;103966310.4;116962099.2;0;129957888            
```
where
--url http://localhost:8080/jolokia

is the agent URL

--mbean java.lang:type=Memory

is the MBean name

--attribute HeapMemoryUsage

is the attribute to monitor

--path used
is an inner path (see "Paths"), which specifies an inner value within a more complex structure. The value HeapMemoryUsage is a composed value (Jav type: CompositeData) which combines multiple memory related data. The complete value can be viewed with jmx4perl:
```
$ jmx4perl http://localhost:8080/jolokia read java.lang:type=Memory HeapMemoryUsage
{
  committed => 85000192,
  init => 0
  max => 129957888,
  used => 15106608,
}
```
--base java.lang:type=Memory/HeapMemoryUsage/max

is the base value for which a relative threshold should be applied. This is a shortcut notation in the format mbean/attribute/path.

--warning 80

is the warning threshold in percent. I.e. a WARNING will be raised by this plugin when the heap memory usage is larger than 80% of the maximal available heap memory for the application server (which is smaller than the available memory of the operating system)

--critical 90

is the critical threshold in percent. If the available heap memory reaches 90% of the available heap, a CRITICAL alert will be returned.
All available command line options are described in "COMMAND LINE".
For more complex checks the usage of a configuration file is recommended. This also allows you to keep your Nagios service definitions small and tidy. E.g. for monitoring the number of request per minute for a certain web application, a predefined check is available:
```
$ check_jmx4perl --url http://localhost:8080/jolokia \
                 --config $J4P/config/tomcat.cfg \
                 --critical 100 \
                 --check tc_servlet_requests \
                 jolokia-agent

OK - 15.00 requests/minute | 'Requests jolokia-agent'=15;5000;100
```
where
--config $J4P/config/tomcat.cfg

is the path to configuration file. There a several predefined checks coming with this distribution, which are documented inline. Look there for some inspiration for what to check.

--critical 100

A threshold von 100, i.e. the checked value must be 100 or less, otherwise a critical alert is raised.

--check tc_servlet_requests

is the name of the check to perform which must be defined in the configuration file

jolokia-agent
is an extra argument used by the predefined check. It is the name of the servlet for which the number of requests should be monitored. To get the name of all registered servlets use jmx4perl list:
```
$ jmx4perl http://localhost:8080/jolokia list | grep j2eeType=Servlet
```
The servlet name is the value of the name property of the listed MBeans.
Configuration files are very powerful and are the recommended way for configuring check_jmx4perl for any larger installation. Features like multi checks are even only available when using a configuration file. The syntax for configuration files are explained in depth in "CONFIGURATION".

Finally, a Nagios service definition needs to be added. For the memory example above, a command for relative checks can be defined:

define command {
   command_name         check_jmx4perl_relative
   command_line         $USER3$/check_jmx4perl \
                            --url $ARG1$ \
                            --mbean $ARG2$ \
                            --attribute $ARG3$ \
                            --path $ARG4$ \
                            --base $ARG5$ \
                            $ARG6$
}

Put this into place where you normally define commands (either in the global Nagios commands.cfg or in a specific commands configuration file in the commands directory). $USER3 is a custom variable and should point to the directory where check_jmx4perl is installed (e.g. /usr/local/bin).

The service definition itself then looks like:

define service {
   service_description    j4p_localhost_memory
   host_name              localhost
   check_command          check_jmx4perl_relative \
                          !http://localhost:8080/jolokia \
                          !java.lang:type=Memory \
                          !HeapMemoryUsage \
                          !used \
                          !java.lang:type=Memory/HeapMemoryUsage/max \
                          !--warning 80 --critical 90
}

Add this section to your service definitions (depending on your Nagios installation). This example adds a service to host localhost for checking the heap memory, raising a WARNING if 80% of the available heap is used and a CRITICAL if more than 90% of the heap memory is occupied.

Installing and using jmx4perl is really that easy. The Nagios configuration in this example is rather simplistic, of course a more flexible Nagios setup is possible. The blog post http://labs.consol.de//blog/jmx4perl/check_jmx4perl-einfache-servicedefinitionen/ (written by Gerhard Lausser) shows some advanced configuration setup. (It is in german, but the automatic translation from http://bit.ly/bgReAs seems to be quite usable).

REFERENCE

This section explains the JMX basics necessary to better understand the usage of check_jmx4perl. It tries to be as brief as possible, but some theory is required to get the link to the Java world.

MBeans

JMX's central entity is an MBean. An MBean exposes management information in a well defined way. Each MBean has a unique name called Object Name with the following structure:

domain:attribute1=value1,attribute2=value2, .....

E.g.

java.lang:type=Memory

points to the MBean which lets you access the memory information of the target server.

Unfortunately, except for so called MXBeans (http://java.sun.com/j2se/1.5.0/docs/api/java/lang/management/package-summary.html) there is no standard naming for MBeans. Each platform uses its own. There used to be a naming standard defined in JSR77 (http://jcp.org/en/jsr/detail?id=77), unfortunately it was never widely adopted.

There are various ways for identifying MBeans on a server:

Use jmx4perl --list to list all registered MBeans. In addition jmx4perl --attributes dumps out all known MBean attributes along with their values. (Be careful, the output can be quite large)
Use j4psh for interactively exploring the JMX namespace.
Use an alias. An alias is a shortcut for an MBean name, predefined by JMX::Jmx4Perl. All known aliases can be shown with jmx4perl aliases. Since each platform can have slightly different MBean names for the same information, this extra level of indirection might help in identifying MBeans. See "Aliases" for more about aliases.
Use a predefined check. check_jmx4perl comes with quite some checks predefined in various configuration files. These are ready for use out of the box. "Predefined checks" are described in an extra section.
Ask your Java application development team for application specific MBean names.

Attributes and Operations

check_jmx4perl can obtain the information to monitor from two sources: Either as MBean attributes or as a return value from JMX operations. Since JMX values can be any Java object, it is important to understand, how check_jmx4perl (or jmx4perl in general) handles this situation.

Simple data types can be used directly in threshold checking. I.e. the following data types can be used directly

Integer
Long
Float
Double
Boolean
String

String and Boolean can be used in string checks only, whereas the others can be used in both, numeric and string checks (see "String checks").

For numeric checks, the threhsholds has to be specified according to the format defined in http://nagiosplug.sourceforge.net/developer-guidelines.html#THRESHOLDFORMAT

Paths

For more complex types, check_jmx4perl provides the concept of so called paths for specifying an inner attribute of a more complex value. A path contains parts separated by slashes (/). It is similar to an XPath expression for accessing parts of an XML document. Each part points to an inner level of a complex object.

For example, the MBean java.lang:type=Memory exposes an attribute called HeapMemoryUsage. This attribute is a compound data type which contains multiple entries. Looking with jmx4perl at this attribute

$ jmx4perl http://localhost:8080/jolokia read java.lang:type=Memory HeapMemoryUsage
{
  committed => 85000192,
  init => 0
  max => 129957888,
  used => 15106608,
}

it can be seen, that there are 4 values coming with the reponse. With a path used one can directly pick the used heap memory usage (8135440 bytes in this case) which then can be used for a threshold check.

$ check_jmx4perl --url http://localhost:8080/jolokia \ 
                 --mbean java.lang:type=Memory \ 
                 --attribute HeapMemoryUsage \
                 --path used \
                 --critical 100000000
OK - [java.lang:type=Memory,HeapMemoryUsage,used] : Value 10136056 in range | ...

Attributes

Attributes are values obtained from MBean properties. Complex values are translated into a JSON structure on the agent side, which works for most types. To access a single value from a complex value, the path mechanism described above can be used. Thresholds can be applied to simple data types only, so for complex attributes a path is required.

Operations

The return values of operations can be used for threshold checking, too. Since a JMX exposed operation can take arguments, these has to be provided as extra arguments on the command line or in the configuration via the Args configuration directive. Due to the agent's nature and the protocol used (JSON), only simple typed arguments like strings, numbers or booleans ("true"/"false") can be used.

Example:

$ check_jmx4perl --url http://localhost:8888/jolokia \
                 --mbean jolokia:type=Runtime \
                 --operation getNrQueriesFor \
                 --critical 10 \
                 "operation" \
                 "java.lang:type=Memory" \
                 "gc"

This example contacts a MBean jolokia:type=Runtime registered by the jolokia agent in order to check for the number of queries for a certain MBean via this agent. For this purpose an JMX operation getNrQueriesFor is exposed which takes three arguments: The type ("operation"/"attribute"), the MBean's ObjectName and the operation/attribute name which was called.

If the operation to be called is an overloaded operation (i.e. an operation whose name exists multiple times on the same MBean but with different parameter types), the argument types must be given within parentheses:

--operation checkUserCount(java.lang.String,java.lang.String)

Aliases

Aliases are shortcut for common MBean names and attributes. E.g. the alias MEMORY_HEAP_MAX specifies the MBean java.lang:type=Memory, the attribute HeapMemoryUsage and the path max. Aliases can be specified with the --alias option or with the configuration directive Alias. Aliases can be translated to different MBean names on different application server. For this check_jmx4perl uses an autodetection mechanism to determine the target platform. Currently this mechanism uses one or more extra server round-trips. To avoid this overhead, the --product option (configuration: Product) can be used to specify the target platform explicitely. This is highly recommended in case you are using the aliasing feature.

Aliases are not extensible and can not take any parameters. All availables aliases can be viewed with

jmx4perl aliases

A much more flexible alternative to aliases are parameterized checks, which are defined in a configuration file. See "CONFIGURATION" for more details about parameterized checks.

Relative Checks

Relative values are often more interesting than absolute numbers. E.g. the knowledge that 140 MBytes heap memory is used is not as important as the knowledge, that 56% of the available memory is used. Relative checks calculate the ratio of a value to a base value. (Another advantage is that Nagios service definitions for relative checks are generic as they can be applied for target servers with different memory footprints).

The base value has to be given with --base (configuration: Base). The argument provided here is first tried as an alias name or checked as an absolute, numeric value. Alternatively, you can use a full MBean/attribute/path specification by using a / as separator, e.g.

... --base java.lang:type=Memory/HeapMemoryUsage/max ...

If one of these parts (the path is optional) contains a slash within its name, the slash must be escaped with a backslash (\/). Backslashes in MBean names are escaped with a double backslash (\\).

Alternatively --base-mbean, --base-attribute and --base-path can be used to specify the parts of the base value separately.

Example:

check_jmx4perl --url http://localhost:8080/jolokia \ 
               --value java.lang:type=Memory/HeapMemoryUsage/used \ 
               --base java.lang:type=Memory/HeapMemoryUsage/max \ 
               --critical 90

check_jmx4perl --url http://localhost:8080/jolokia \ 
               --value java.lang:type=Memory/HeapMemoryUsage/used \ 
               --base-mbean java.lang:type=Memory \
               --base-attribute HeapMemoryUsage \
               --base-path max \ 
               --critical 90

This check will trigger a state change to CRITICAL if the used heap memory will exceed 90% of the available heap memory.

Incremental Checks

For some values it is worth monitoring the increase rate (velocity). E.g. for threads it can be important to know how fast threads are created.

Incremental checks are switched on with the --delta option (configuration: Delta). This option takes an optional argument which is interpreted as seconds for normalization.

Example:

check_jmx4perl --url http://localhost:8080/jolokia \ 
               --mbean java.lang:type=Threading \ 
               --attribute TotalStartedThreadCount \ 
               --delta 60 \ 
               --critical 5

This will fail as CRITICAL if more than 5 threads are created per minute (60 seconds). Technically check_jmx4perl uses the history feature of the jolokia agent deployed on the target server. This will always store the result and the timestamp of the last check on the server side and returns these historical values on the next check so that the velocity can be calculated. If no value is given for --delta, no normalization is used. In the example above, without a normalization value of 60, a CRITICAL is returned if the number of threads created increased more than 5 between two checks.

--delta doesn't work yet with --base (e.g. incremental mode for relative checks is not available).

String checks

In addition to standard numerical checks, direct string comparison can be used. This mode is switched on either explicitely via --string (configuration: String) or by default implicitely if a heuristics determines that a value is non-numeric. Numeric checking can be enforced with the option --numeric (configuration: Numeric).

For string checks, --critical and --warning are not treated as numerical values but as string types. They are compared literally against the value retrieved and yield the corresponding Nagios status if matched. If the threshold is given with a leading !, the condition is negated. E.g. a --critical '!Running' returns CRITICAL if the value not equals to Running. Alternatively you can also use a regular expression by using qr/.../ as threshold value (substitute ... with the pattern to used for comparison). Boolean values are returned as true or false strings from the agent, so you can check for them as well with this kind of string comparison.

No performance data will be generated for string checks by default. This can be switched on by providing --perfdata on (or "PerfData on" in the configuration). However, this probably doesn't make much sense, though.

Output Tuning

The output of check_jmx4perl can be highly customized. A unit-of-measurement can be provided with the option --unit (configuration: Unit) which specifies how the the attribute or an operation's return value should be interpreted. The units available are

B  - Byte
KB - Kilo Byte
MB - Mega Byte
GB - Giga Byte
TB - Terra Byte

us - Microseconds
ms - Milliseconds
s  - Seconds
m  - Minutes
h  - Hours
d  - Days

The unit will be used for performance data as well as for the plugin's output. Large numbers are converted to larger units automatically (and reverse for small number that are smaller than 1). E.g. 2048 KB is converted to 2 MB. Beautifying by conversion is only performed for the plugin output, not for the performance data for which no conversions happens at all.

Beside unit handling, you can provide your own label for the Nagios output via --label. The provided option is interpreted as a pattern with the following placeholders:

%v   the absolute value 
%f   the absolute value as floating point number
%r   the relative value as percentage (--base)
%q   the relative value as ratio of value to base (--base)
%u   the value's unit for the output when --unit is used (after shortening)
%w   the base value's unit for the output when --unit is used (after shortening)
%b   the absolut base value as it is used with --base 
%c   the Nagios exit code in the Form "OK", "WARNING", "CRITICAL" 
     or "UNKNOWN"
%t   Threshold value which failed ("" when the check doesn't fail)
%n   name, either calulated automatically or given with --name
%d   the delta value used for normalization when using incremental mode
%y   WARNING threshold as configured
%z   CRITICAL threshold as configured

Note that %u and %w are typically not the same as the --unit option. They specify the unit after the conversion for the plugin output as described above. You can use the same length modifiers as for sprintf to fine tune the output.

Example:

check_jmx4perl --url http://localhost:8888/jolokia \
               --alias MEMORY_HEAP_USED \
               --base MEMORY_HEAP_MAX \ 
               --critical :80 \
               --label "Heap-Memory: %.2r% used (%.2v %u / %.2b %w)" \
               --unit B

will result in an output like

OK - Heap-Memory: 3.48% used (17.68 MB / 508.06 MB) | '[MEMORY_HEAP_USED]'=3.48%;;:80

Security

Since the jolokia-agent is usually a simple war-file, it can be secured as any other Java Webapplication. Since setting up authentication is JEE Server specific, a detailed instruction is beyond the scope of this document. Please refer to your appserver documentation, how to do this. At the moment, check_jmx4perl can use Basic-Authentication for authentication purposes only.

In addition to this user/password authentication, the jolokia-agent uses a policy file for fine granular access control. The policy is defined with an XML file packaged within the agent. In order to adapt this to your needs, you need to extract the war file, edit it, and repackage the agent with a policy file. A future version of jmx4perl might provide a more flexible way for changing the policy.

In detail, the following steps are required:

Download jolokia.war and a sample policy file jolokia-access.xml into a temporary directory:
```
$ jolokia
$ jolokia --policy
```
Edit the policy according to your needs.
```
$ vi jolokia-access.xml
```
Repackage the war file
```
$ jolokia repack --policy jolokia.war
```
Deploy the agent jolokia.war as usual

The downloaded sample policy file jolokia-access.xml contains inline documentation and examples, so you can easily adapt it to your environment.

Restrictions can be set to on various parameters :

Client IP address

Access to the jolokia-agent can be restricted based on the client IP accessing the agent. A single host, either with hostname or IP address can be set or even a complete subnet.

Example:

<remote>
  <host>127.0.0.1</host>
  <host>10.0.0.0/16</host>
</remote>

Only the localhost or any host in the subnet 10.0 is allowed to access the agent. If the <remote> section is missing, access from all hosts is allowed.

Commands

The access can be restricted to certain commands.

Example:

<commands>
  <command>read</command>
</commands>

This will only allow reading of attributes, but no other operation like execution of operations. If the <commands> section is missing, any command is allowed. The commands known are

read: Read an attribute
write: Write an attribute (used by check_jmx4perl only when using incremental checks)
exec: Execution of an operation
list: List all MBeans (not used by check_jmx4perl)
version: Version command (not used by check_jmx4perl)
search: Search for MBean (not used by check_jmx4perl)

Specific MBeans

The most specific policy can be put on the MBeans themselves. For this, two sections can be defined, depending on whether a command is globaly enabled or denied.

<allow>

The <allow> section is used to switch on access for operations and attributes in case read, write or exec are globally disabled (see above). Wildcards can be used for MBean names and attributes/and operations.

Example:

<allow>
  <mbean>
    <name>jolokia:*</name>
    <operation>*</operation>
    <attribute>*</attribute>
  </mbean>
  <mbean>
    <name>java.lang:type=Threading</name>
    <operation>findDeadlockedThreads</operation>
  </mbean>
  <mbean>
  <name>java.lang:type=Memory</name>
    <attribute mode="read">Verbose</attribute>
  </mbean>

</allow>

This will allow access to all operation and attributes of all MBeans in the jolokia: domain and to the operation findDeadlockedThreads on the MBean java.lang:type=Threading regardless whether the read or exec command is enabled globally. The attribute Verbose on java.lang:type=Memory is allowed to be read, but cannot be written (if the mode attribute is not given, both read and write is allowed by default).

<deny>

The <deny> section forbids access to certain MBean's operation and/or attributes, even when the command is allowed globally.

Example:

<deny>
  <mbean>
    <!-- Exposes user/password of data source, so we forbid this one -->
    <name>com.mchange.v2.c3p0:type=PooledDataSource*</name>
    <attribute>properties</attribute>
  </mbean>
</deny>

This will forbid the access to the specified attribute, even if read is allowed globally. If there is an overlap between <allow> and <deny>, <allow> takes precedence.

Proxy mode

check_jmx4perl can be used in an agentless mode as well, i.e. no jolokia-agent needs to deployed on the target server. The setup for the agentless mode is a bit more complicated, though:

The target server needs to export its MBeans via JSR-160. The configuration for JMX export is different for different JEE Server. http://labs.consol.de has some cooking recipes for various servers (JBoss, Weblogic).
A dedicated proxy server needs to be setup on which the jolokia.war gets deployed. This can be a simple Tomcat or Jetty servlet container. Of course, an already existing JEE Server can be used as proxy server as well.
For using check_jmx4perl the target JMX URL for accessing the target server is required. This URL typically looks like
```
service:jmx:rmi:///jndi/rmi://host:9999/jmxrmi
```
but this depends on the server to monitor. Please refer to your JEE server's documentation for how the export JMX URL looks like.
check_jmx4perl uses the proxy mody if the option --target (configuration: <Target>) is provided. In this case, this Nagios plugin contacts the proxy server specified as usual with --url (config: Url in Server section) and put the URL specified with --target in the request. The agent in the proxy then dispatches this request to the real target and uses the JMX procotol specified with in the target URL. The answer received is then translated into a JSON response which is returned to check_jmx4perl.

Example:
```
check_jmx4perl --url http://proxy:8080/jolokia \
               --target service:jmx:rmi:///jndi/rmi://jeeserver:9999/jmxrmi
               --alias MEMORY_HEAP_USED
               --base MEMORY_HEAP_MAX
               --critical 90
```
Here the host proxy is listening on port 8080 for jolokia requests and host jeeserver exports its JMX data via JSR-160 over port 9999. (BTW, proxy can be monitored itself as usual).

So, what mode is more appropriate ? Both, the agent mode and the proxy mode have advantages and disadvantages.

Advantages

No agent needs to be installed on the target server. This might be useful for policy reasons.
Compared to other Nagios JMX plugin's no JVM startup is required since the proxy server is already running.

Disadvantages

It takes two hops to get to the target server
Exporting JMX via JSR-160 is often not that easy as it may seem. (See post series on remote JMX on labs.consol.de)
Some features like merging of MBean Servers are not available in proxy mode. (i.e you need to know in advance which MBean-Server on the target you want to contact for a certain MBean, since this information is part of the JMX URL)
Bulk request needs to be detangled into multiple JMX request since JSR-160 doesn't know anything about bulk requests.
jmx4perl's fine granular security policy is not available, since JSR-160 JMX is an all-or-nothing thing. (except you are willing to dive deep into Java Security stuff)
For JSR-160 objects to be transferable to the proxy, the proxy needs to know about the Java types and those types must be serializable. If this is not the case, the proxy isn't able to collect the information from the target. So only a subset of MBeans can be monitored this way.

The agent protocol is more flexible since it translates the data into a JSON structure before putting it on the wire.

To summarize, I would always recommend the agent mode over the proxy mode except when an agentless operation is required (e.g. for policy reasons).

COMMAND LINE

The pure command line interface (without a configuration file) is mostly suited for simple checks where the predefined defaults are suitable. For all other use cases, a configuration file fits better.

check_jmx4perl knows about the following command line options:

--url (-u)

The URL for accessing the target server (or the jolokia-proxy server, see "Proxy Mode" for details about the JMX proxy mode)

Example:

--url http://localhost:8080/jolokia

--mbean (-m)

Object name of MBean to access

Example:

--mbean java.lang:type=Runtime

--attribute (-a)

A MBean's attribute name. The value of this attribute is used for threshold checking.

Example:

--attribute Uptime

--operation (-o)

A MBean's operation name. The operation gets executed on the server side and the return value is used for threshold checking. Any arguments required for this operation has to be given as additional arguments to check_jmx4perl. See "Attributes and Operations" for details.

Example:

check_jmx4perl ... --mbean java.lang:type=Threading \
                   --operation getThreadUserTime 1

Operation getThreadUserTime takes a single argument the thread id (a long) which is given as extra argument.

--path (-p)

Path for extracting an inner element from an attribute or operation return value. See "Paths" for details about paths.

Example:

--path used

--value

Shortcut for giving --mbean, --attribute and --path at once.

Example:

--value java.lang:type=Memory/HeapMemoryUsage/used

Any slash (/) in the MBean name must be escaped with a backslash (\/). Backslashes in names has to be escaped as \\.

--base (-b)

Switches on relative checking. The value given points to an attribute which should be used as base value and has to be given in the shortcut notation described above. Alternatively, the value can be an absolute number or an alias name ("Aliases") The threshold are the interpreted as relative values in the range [0,100]. See "Relative Checks" for details.

Example:

--base 100000
--base java.lang:type=Memory/HeapMemoryUsage/max
--base MEMORY_HEAP_MAX

--delta (-d)

Switches on incremental checking, i.e. the increase rate (or velocity) of an attribute or operation return value is measured. The value given here is used for normalization (in seconds). E.g. --delta 60 normalizes the velocity to 'growth per minute'. See "Incremental Checks" for details.

--string

Forces string checking, in which case the threshold values are compared as strings against the measured values. See "String checks" for more details. By default, a heuristic based on the measured value is applied to determine, whether numerical or string checking should be use

Example:

 --string --critical '!Running'

--numeric

Forces numeric checking, in which case the measured valued are compared against the given thresholds according to the Nagios developer guideline specification (http://nagiosplug.sourceforge.net/developer-guidelines.html#THRESHOLDFORMAT)

Example:

--numeric --critical ~:80

--null

The value to be used in case the attribute or the operation's return value is null. This is useful when doing string checks. By default, this value is "null".

Example:

--null "no deadlock" --string --critical "!no deadlock"

--name (-n)

Name to be used for the performance data. By default a name is calculated based on the MBean's name and the attribute/operation to monitor.

Example:

--name "HeapMemoryUsage"

--label (-l)

Label for using in the plugin output which can be a format specifier as described in "Output Tuning".

Example:

--label "%.2r% used (%.2v %u / %.2b %w)"

--perfdata

Switch off ("off") or on ("on") performance data generation. Performance data is generated by default for numerical checks and omitted for string based checks. For relative checks, if the value is '%' then performance data is appended as relative values instead of absolute values.

--unit

Natural unit of the value measured. E.g. when measuring memory, then the memory MXBean exports this number as bytes. The value given here is used for shortening the value's output by converting to the largest possible unit. See "Output Tuning" for details.

Example:

--alias MEMORY_HEAP_USED --unit B

--critical (-c)

Critical threshold. For string checks, see "String checks" for how the critical value is interpreted. For other checks, the value given here should conform to the specification defined in http://nagiosplug.sourceforge.net/developer-guidelines.html#THRESHOLDFORMAT.

Example:

--critical :90

--warning (-w)

Warning threshold, which is interpreted the same way as the --critical threshold (see above). At least a warning or critical threshold must be given.

--alias

An alias is a shortcut for an MBean attribute or operation. See "Aliases" for details.

Example:

--alias RUNTIME_UPTIME

--product

When aliasing is used, check_jmx4perl needs to known about the target server type for resolving the alias. By default it used an autodetection facility, which at least required an additional request. To avoid this, the product can be explicitely specified here

Example:

--product jboss

--user, --password

User and password needed when the agent is secured with Basic Authentication. By default, no authentication is used.

--timeout (-t)

How long to wait for an answer from the agent at most (in seconds). By default, the timeout is 180s.

--method

The HTTP metod to use for sending the jmx4perl request. This can be either get or post. By default, an method is determined automatically. get for simple, single requests, post for bulk request or requests using a JMX proxy.

--proxy

A HTTP proxy server to use for accessing the jolokia-agent.

Example:

--proxy http://proxyhost:8001/

--legacy-escape

When the deployed Jolokia agent's version is less than 1.0, then this option should be used since the escape scheme as changed since version 1.0. This option is only important for MBeans whose names contain slashes. It is recommended to upgrade the agent to a post 1.0 version, though.

--target, --target-user, --target-password

Switches on jolokia-proxy mode and specifies the URL for accessing the target platform. Optionally, user and password for accessing the target can be given, too. See "Proxy Mode" for details.

Example:

--target service:jmx:rmi:///jndi/rmi://bhut:9999/jmxrmi

--config

Specifies a configuration file from where server and check definitions can be obtained. See "CONFIGURATION" for details about the configuration file's format.

Example:

--config /etc/jmx4perl/tomcat.cfg

--server

Specify a symbolic name for a server connection. This name is used to lookup a server in the configuration file specified with --config

Example:

servers.cfg:
  <Server tomcat>
     Url http://localhost:8080/jolokia
     User roland
     Password fcn
  </Server>

  --config /etc/jmx4perl/servers.cfg --server tomcat

See "CONFIGURATION" for more about server definitions.

--check

The name of the check to use as defined in the configuration file. See "CONFIGURATION" about the syntax for defining checks and multi checks. Additional arguments for parameterized checks should be given as additional arguments on the command line. Please note, that checks specified with --check have precedence before checks defined explicitely on the command line.

Example:

--config /etc/jmx4perl/tomcat.cfg --check tc_servlet_requests jolokia-agent

--version

Prints out the version of this plugin

--verbose (-v)

Enables verbose output during the check, which is useful for debugging. Don't use it in production, it will confuse Nagios.

--doc, --help (-h), --usage (-?)

--usage give a short synopsis, --help prints out a bit longe usage information.

--doc prints out this man page. If an argument is given, it will only print out the relevant sections. The following sections are recognized:

tutorial: A 5 minute quickstart
reference: Reference manual explaining the various operational modes.
options: Command line options available for check_jmx4perl
config: Documentation for the configuration syntax

CONFIGURATION

Using check_jmx4perl with a configuration file is the most powerful way for defining Nagios checks. A simple configuration file looks like

# Define server connection parameters
<Server tomcat>
   Url = http://localhost:8080/jolokia
</Server>

# A simple heap memory check with a critical threshold of 
# 90% of the maximal heap memory. 
<Check memory_heap>     
  Value = java.lang:type=Memory/HeapMemoryUsage/used
  Base = java.lang:type=Memory/HeapMemoryUsage/max
  Unit = B
  Label = Heap-Memory: %.2r% used (%.2v %u / %.2b %u)
  Name = Heap
  Critical = 90
</Check>

A configuration file is provided on the command line with the option --config. It can be divided into two parts: A section defining server connection parameters and a section defining the checks themselves.

<Server>

With <Server name> the connection parameters for a specific server is defined. In order to select a server the --server name command line option has to be used. Within a <Server> configuration element, the following keys can be used:

Url

The URL under which the jolokia agent can be reached.

User, Password

If authentication is switched on, the user and the credentials can be provided with the User and Password directive, respectively. Currently only Basic Authentication is supported.

Product

The type of application server to monitor. This configuration can speed up checks significantly, but only when aliases are used. By default when using aliases, check_jmx4perl uses autodetection for determine the target's platform. This results in at least one additional HTTP-Request. This configuration does not has any effect when MBeans are always used with their full name.

Proxy

A HTTP Proxy URL and credentials can be given with the <Proxy> sub-section. Example:

<Server>
....
  <Proxy>
    Url = http://proxy.company.com:8001
    User = woody
    Password = buzz
  </Proxy>
</Server>

Url: The proxy URL
User, Password: Optional user and credentials for accessing the proxy

Target

With this directive, the JMX-Proxy mode can be switched on. As described in section "Proxy mode", check_jmx4perl can operate in an agentless mode, where the agent servlet is deployed only on an intermediated, so called JMX-Proxy server, whereas the target platform only needs to export JMX information in the traditional way (e.g. via JSR-160 export). This mode is especially useful if the agent is not allowed to be installed on the target platform. However, this approach has some drawbacks and some functionality is missing there, so the agent-mode is the recommended way. A sample JMX-Proxy configuration looks like:

<Target>
   Url = service:jmx:rmi:///jndi/rmi://tessin:6666/jmxrmi
   User = max
   Password = frisch 
</Target>

For a discussion about the advantages and disadvantages of the JMX-Proxy mode, please have a look at http://labs.consol.de/ which contains some evaluations of this mode for various application servers (e.g. JBoss and Weblogic).

Url: The JMX-RMI Url to access the target platform.
User, Password: User and password for authentication against the target server.

Single Check

With <Check> a single check can be defined. It takes any option available also available via the command line. Each check has a name, which can be referenced from the commandline with the option --check name.

Example:

<Check memory_heap>
  Value = java.lang:type=Memory/HeapMemoryUsage/used
  Base = java.lang:type=Memory/HeapMemoryUsage/max
  Label = Heap-Memory:
  Name = Heap
  Critical = 90
</Check>

The <Check> section knows about the following directives:

Mbean

The ObjectName of the MBean to monitor.

Attribute

Attribute to monitor.

Operation

Operation, whose return value should be monitored. Either Attribute or Operation should be given, but not both. If the operation takes arguments, these need to be given as additional arguments to the check_jmx4perl command line call. In the rare case, you need to call an overloaded operation (i.e. an operation whose name exists multiple times on the same MBean but with different parameter types), the argument types can be given within parentheses:

<Check>
   ....
   Operation = checkUserCount(java.lang.String,java.lang.String)
   ...
</Check>

Argument

Used for specifying arguments to operation. This directive can be given multiple times for multiple arguments. The order of the directive determine the order of the arguments.

<Check>
   ....
   Operation checkUserCount(java.lang.String,java.lang.String)
   Argument  Max
   Argument  Morlock    
</Check>

Alias

Alias, which must be known to check_jmx4perl. Use jmx4perl aliases to get a list of all known aliases. If Alias is given as configuration directive, Operation and/or Attribute is ignored. Please note, that using Alias without Product in the server section leads to at least one additional HTTP request.

Path

Path to apply to the attribute or operation return value. See "Paths" for more information about paths.

Value

Value is a shortcut for specifying MBean, Attribute and Path at once. Simply concatenate all three parts via / (the Path part is optional). Slashes within MBean names needs to be escaped with a \ (backslash). Example:

Value = java.lang:type=Memory/HeapMemoryUsage/used

is equivalent to

MBean = java.lang:type=Memory
Attribute = HeapMemoryUsage
Path = used

Base

Switches on relative checks. See "Relative Checks" for more information about relative checks. The value specified with this directive defines the base value against which the relative value should be calculated. The format is the same as for Value:

Base = java.lang:type=Memory/HeapMemoryUsage/max

For relative checks, the Critical and Warning Threshold are interpreted as a value between 0% and 100%.

BaseMBean, BaseAttribute and BasePath

As an alternative to specifying a base value in a combined fashion the different parts can be given separately. BaseMBean and BaseAttribute switches on relative checks and specifies the base value. An optional BasePath can be used to provide the path within this base value.

The example above can be also written as

BaseMBean = java.lang:type=Memory
BaseAttribute = HeapMemoryUsage
BasePath = max

Delta

Switches on incremental mode as described in section "Incremental Checks". The value given is used for normalization the increase rate. E.g.

Delta = 60

measures the growth rate per minute (60 seconds). If no value is given, the absolute increase between two checks is used.

Numeric

This directive switches on numeric mode, i.e. the given threshold values are compared numerically against the returned JMX value. By default, the check mode is determined by a heuristic algorithm.

String

String checks, which are switched on with this directive, are useful for non-numeric thresholds. See "String checks" for more details.

Name

The name to be used in the performance data. By default, a name is calculated based on the MBean and attribute/operation name.

MultiCheckPrefix

If this check is used within a multi check, this prefix is used to identify this particular check in the output of a multicheck. It can be set to an empty string if no prefix is required. By default the name as configured with Name is used.

Label

Format for setting the plugin output (not the performance data, use Name for this). It takes a printf like format string which is described in detail in "Output Tuning".

PerfData

By default, performance data is appended for numeric checks. This can be tuned by setting this directive to "false" (or "0", "no", "off") in which case performance data is omitted. If using this in a base check, an inherited check can switch performance data generation back on with "true" (or "1", "yes", "on")

For relative checks, the value can be set to '%'. In this case, performance data is added as relative values instead of the absolute value measured.

Unit

This specifies how the return value should be interpreted. This value, if given, must conform to the unit returned by the JMX attribute/operation. E.g. for java.lang:type=Memory/HeapMemoryUsage/used unit, if set, must be B since this JMX call returns the used memory measured in bytes. The value given here is only used for shortening the plugin's output automatically. For more details and for what units are available refer to section "Output Tuning".

Critical

Specifies the critical threshold. If String is set (or the heuristics determines a string check), this should be a string value as described in "String checks". For relative checks, this should be a relative value in ther range [0,100]. Otherwise, it is a simple numeric value which is used as threshold. For numeric checks, the threshhold can be given in the format defined at http://nagiosplug.sourceforge.net/developer-guidelines.html#THRESHOLDFORMAT.

Warning

Defines the warning threshold the same way as described for the Critical threshold.

Null

Replacement value when an attribute is null or an operation returns a null value. This value then can be used in string checks in order to check against null values. By default, this value is "null".

Method

HTTP Method to use for the check. Available values are GET or POST for GET or POST HTTP-Requests, respectively. By default a method is determined automatically. The value can be given case insensitively.

Use

In order to use parent checks, this directive specifies the parent along with any parameters passed through. For example,

Use = memory_relative_base(80,90),base_label

uses a parent check named memory_relative_base, which must be a check defined in the same configuration file (or an imported on). Additionally, the parameters 80 and 90 are passed to this check (which can be accessed there via the argument placeholders $0 and $1). See "Parent checks" and "Parameterized checks" for more information about check inheritance.

Multiple parents can be given by providing them in a comma separated list.

Script

For complex checks which can not be realized with the configurations described until yet, it is possible to use a Perl script snippet to perfrom arbitrary logic. The content of this script is typically provided as an HERE-document (see example below). It comes with a predefined variable $j4p which is an instance of JMX::Jmx4Perl so that it can be used for a flexible access to the server. Note that this scriptlet is executed separately and doesn't not benefit from the optimization done for bulk or relative checks. Check parameters can be accessed as ${0}, ${1}, .. but since these are also valid Perl variables (and hence can be overwritten accidentially), it is recommended to assign them to local variable before using them. In summary, script based checks are powerful but might be expensive.

Example:

Script <<EOT

my $pools = $j4p->search("java.lang:type=MemoryPool,*");
my @matched_pools;
my $pattern = "${0}";
for my $pool (@$pools) {   
   push @matched_pools,$pool if $pool =~ /$pattern/;   
}
return $j4p->get_attribute($matched_pools[0],"Usage","used");

EOT

Includes

Checks can be organized in multiple configuration files. To include another configuration file, the include directive can be used:

include tomcat.cfg
include threads.cfg
include memory.cfg

If given as relative path, the configuration files are looked up in the same directory as the current configuration file. Absolute paths can be given, too.

Parent checks

With check_jmx4perl parent checks it is possible to define common base checks, which are usable in various sub-checks. Any <Check> can be a parent check as soon as it is referenced via a Use directive from within another check's definition. When a check with a parent check is used, its configuration is merged with this from the parent check with own directives having a higher priority. Parent checks can have parent checks as well (and so on).

For example, consider the following configuration:

<Check grand_parent>
   Name grand_parent
   Label GrandPa
   Critical 10
</Check>

<Check parent_1>
   Use grand_parent
   Name parent_1
   Critical 20
</Check>

<Check parent_2>
   Name parent_2
   Warning 20
</Check> 

<Check check>
   Use parent_1,parent_2
   Warning 40
</Check>

In this scenario, when check check is used, it has a Name "parent_2" (last parent check in Use), a Label "GrandPa" (inherited from grand_parent via parent_1), a Critical 20 (inherited from parent_1) and a Warning 40 (directly give in the check definition).

A parent value of a configuration directive can be refered to with the placeholder $BASE. For example:

<Check parent>
  Name Parent
</Check>

<Check check>
  Use parent
  Name Child: $BASE
</Check>

This will lead to a Name "Child: Parent" since $BASE is resolved to the parent checks valus of Name, "Parent" in this case. The base value is searched upwards in the inheritance hierarchy (parent, grand parent, ...) until a value is found. If nonen is found, an empty string is used for $BASE.

Parameterized checks

Checks can be parameterized, i.e. they can take arguments which are replaced in the configuration during runtime. Arguments are used in check definition via the positional format $0, $1, .... (e.g. $0 is the first argument given). Arguments can either be given on the command line as extra arguments to check_jmx4perl or within the Use directive to provide arguments to parent checks.

Example:

<Check parent>
  Name $0
  Label $1
</Check>

<Check child_check>
  Use parent($0,"Check-Label")   
  ....
</Check>

$ check_jmx4perl --check child_check .... "Argument-Name"
OK - Check-Label | 'Argument-Name'= ....

As it can be seen in this example, arguments can be propagated to a parent check. In this case, $0 from the command line (Argument-Name) is passed through to the parent check which uses it in the Name directive. $1 from the parent check is replaced with the value "Check-Label" given in the Use directive of the child check.

Parameters can have default values. These default values are taken in case an argument is missing (either when declaring the parent check or missing from the command line). Default values are specified with ${arg-nr:default}. For example,

<Check relative_base>
  Label = %.2r% used (%.2v %u / %.2b %w)
  Critical = ${0:90}
  Warning = ${1:80}
</Check>

defines a default value of 90% for the critical threshold and 80% for the warning threshold. If a child check uses this parent definition and only wants to ommit the first parameter (but explicitely specifying the second parameter) it can do so by leaving the first parameter empty:

<Check child>
   Use relative_base(,70)
</Check>

Multichecks

Multiple checks can be combined to a single MultiCheck. The advantage of a multi check is, that multiple values can be retrieved from the server side with a single HTTP request. The output is conformant to Nagios 3 multiline format. It will lead to a CRITICAL value as soon as one check is critical, same for WARNING. If both, CRITICAL and WARNING is triggered by two or more checks, then a CRITICAL is taken.

A multi-check can be defined with the directive <MultiCheck>, which contain various references to other <Check> definitions or other multi check definitions.

Example:

<MultiCheck all>
  MultiCheck memory
  MultiCheck threads
</MultiCheck>

<MultiCheck memory>
  Check memory_heap($0,80)
  Check memory_pool_base("CMS Perm Gen",90,80)
</MultiCheck>

<MultiCheck threads>
  Check thread_inc
  Check thread_deadlock
</MultiCheck>

Here a multi check group memory has been defined with reference to two checks, which must exist somewhere else in the configuration file. As it can be seen, parameters can be given through to the check in the usual way (literally or with references to command line arguments). The group all combines the two groups memory and thread, containing effectively four checks.

A multi-check is referenced from the command line like any other check:

$ check_jmx4perl .... --check all 90

(90 is the argument which replaces $0 in the definition above).

The summary label in a multi check can be configured, too.

Example:

<MultiCheck memory>
  SummaryOk All %n checks are OK
  SummaryFailure %e of %n checks failed [%d]
  ...
</MultiCheck>

These format specifiers can be used:

%n        Number of all checks executed 
%e        Number of failed checks
%d        Details which checks failed

Predefined checks

check_jmx4perl comes with a collection of predefined configuration for various application servers. The configurations can be found in the directory config within the toplevel distribution directory. The configurations are fairly well documented inline.

common.cfg

Common check definitions, which can be used as parents for own checks. E.g. a check relative_base can be used as parent for getting a nicely formatted output message.

memory.cfg

Memory checks for heap and non-heap memoy as well as for various memory pools. Particularly interesting here is the so called Perm Gen pool as it holds the java type information which can overflow e.g after multiple redeployments when the old classloader of the webapp can't be cleared up by the garbage collector (someone might still hold a reference to it).

threads.cfg

Checks for threads, i.e. checking for the tread count increase rate. A check for finding out deadlocks (on a JDK 6 VM) is provided, too.

jetty.cfg

Various checks for jetty like checking for running servlets, thread count within the app server, sessions (number and lifing time) or requests per minute.

tomcat.cfg

Mostly the same checks as for jetty, but for tomcat as application server.

LICENSE

This file is part of jmx4perl.

Jmx4perl is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 2 of the License, or (at your option) any later version.

jmx4perl is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with jmx4perl. If not, see <http://www.gnu.org/licenses/>.

AUTHOR

roland@cpan.org

To install JMX::Jmx4Perl, copy and paste the appropriate command in to your terminal.

cpanm

cpanm JMX::Jmx4Perl

CPAN shell

perl -MCPAN -e shell
install JMX::Jmx4Perl

For more information on module installation, please visit the detailed CPAN module installation guide.

	Global
`s`	Focus search bar
`?`	Bring up this help dialog

	GitHub
`g` `p`	Go to pull requests
`g` `i`	go to github issues (only if github is preferred repository)

	POD
`g` `a`	Go to author
`g` `c`	Go to changes
`g` `i`	Go to issues
`g` `d`	Go to dist
`g` `r`	Go to repository/SCM
`g` `s`	Go to source
`g` `b`	Go to file browse

	Search terms
module: (e.g. module:Plugin)
distribution: (e.g. distribution:Dancer auth)
author: (e.g. author:SONGMU Redis)
version: (e.g. version:1.00)

NAME

SYNOPSIS

DESCRIPTION

TUTORIAL

REFERENCE

MBeans

Attributes and Operations

Paths

Attributes

Operations

Aliases

Relative Checks

Incremental Checks

String checks

Output Tuning

Security

Client IP address

Commands

Specific MBeans

Proxy mode

Advantages

Disadvantages

COMMAND LINE

CONFIGURATION

<Server>

Single Check

Includes

Parent checks

Parameterized checks

Multichecks

Predefined checks

common.cfg

memory.cfg

threads.cfg

jetty.cfg

tomcat.cfg

LICENSE

AUTHOR

Module Install Instructions