This document discusses using Nagios, an open source tool, to monitor WebObjects applications. Nagios can monitor things like HTTP requests, disk space, and load using default plugins. It also allows writing custom plugins to monitor Java-specific metrics via JMX, such as memory usage, threads, and classes loaded. The document demonstrates enabling JMX in a WebObjects app, using the check_jmx Nagios plugin to check JMX metrics, and graphing metrics over time using tools like PNP4Nagios or NagiosGraph.
2. Why monitoring?
• Because you want to find out problems before your customers
find them!
• This is not about finding bugs in your app, more about finding
deployment problems.
• Have to be notified about any problems, including OS, hardware
and Web server problems.
3. Nagios
• My favorite tool for monitoring.
• Been around since 1999 (was called Netsaint back in the days).
• Open source (C and PERL code).
• Works fine on Linux and OS X (use MacPorts to install on OS
X).
4. Nagios
• Can monitor many things with the default plugins : HTTP, SNMP,
disk space, load, etc.
• Can do remote checks (checks runs on the "client", essential for
things like disk space or load).
• Tons of plugins on Nagios Exchange, including for RDMBS like
Oracle or MySQL.
• Easy to write plugins, be it in PERL, Bash, C or even Java.
5. Nagios HTTP check
• You can use the default "check_http" plugin.
• Will check its state based on content's response, timeout, or
HTTP response code (404, 500, etc.)
• Can post data (might be useful for REST services).
• Can specify a specific HTTP method (HEAD, OPTIONS, TRACE,
PUT, DELETE, POST, GET).
• Can even check if your SSL certificate is still valid!
7. Java-specific checks
• Checks with check_http are fine... but won't find some problems
like "Out of memory" errors.
• You can use JMX for that!
8. Java-specific checks
• With JMX, you can monitor things like heap memory usage,
number of threads, number of loaded classes and CPU usage.
• Works fine on Java 5 and 6.
• Easy to enable.
9. Enabling JMX in your app
bash-3.2# cp /System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home/lib/management/
jmxremote.password.template /Library/WebObjects/jmxremote.password
bash-3.2# chown appserver /Library/WebObjects/jmxremote.password
bash-3.2# chmod 600 /Library/WebObjects/jmxremote.password
In your apps launch arguments:
-Dcom.sun.management.jmxremote.port=XXXX
-Dcom.sun.management.jmxremote.authenticate=true
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.password.file=/Library/WebObjects/jmxremote.password
10. WO-specific checks
• With WO 5.4, the stats available from WOStats are also available
by JMX.
• It give you access to data like memory usage, average session
memory, and average requests per session.
• You need to enable JMX + add one line in your Application class
constructor.
12. Enabling WO JMX in your app
import com.webobjects.appserver.WOStatisticsStore;
public Application() {
...
registerMBean((Object)statisticsStore(), getJMXDomain(), WOStatisticsStore.class.getName());
}
13. check_jmx
• check_jmx is a Nagios plugin to connect to JMX-enabled apps.
• Written in Java, source code is available.
• Sadly, it can only check results of type number or string, no
hashmap support.
15. Finding EOF deadlocks
• Create two DirectActions : one that use EOF, the other pure
JDBC.
• Make sure the DA returns the same result everytime.
• Call the DA with check_http, and validate the result.
• If the JDBC check is good, but not the EOF one, good chance
that you have a EOF deadlock.
16. Checking wotaskd config
• Nagios plugin that I wrote to check for various settings, based on
the /admin direct actions available in Wonder's JavaMonitor.
• Useful to check settings, like if auto recover or refuse new
sessions is off, number of deaths have reached a certain level or
that the app is not running.
• Work in progress, will release it in late September.
17. Graphing
• Would be nice to graph some data, like memory usage.
• Lot of tools can do this (Cacti, SNMP tools, etc.), but you can do
it with Nagios too.
• Graphing and performance data are useful when stress loading
your application with JMeter.
• Look at PNP4Nagios or NagiosGraph if you want graphing.