Method 2 - WrapperStartStopApp Integration (Linux / UNIX)

Overview

The Method 2 is to use the WrapperStartStopApp helper class. This method provides a way to integrate with applications like Tomcat, which are started using one class and then stopped using another class.

Typically, this kind of application will open a server socket on startup whose job is to wait for a connection that triggers a shutdown. The shutdown, or "stop" class, triggers the shutdown by connecting to application when launched. The Wrapper works with this kind of application by starting up the application, as in the Method 1, using the "start" class and then calling the main method of the "stop" class when it is time for the application to be shutdown.

When integrating with this Method 2 (WrapperStartStopApp helper class), the WrapperStartStopApp class replaces an application's main class. This gives the WrapperStartStopApp class a chance to immediately initialize the WrapperManager and register the JVM with the Wrapper. The WrapperStartStopApp class then manages all interaction with the Wrapper as well as the life-cycle of an application. When the Wrapper sends a start message to the JVM via the WrapperManager, the main method of the application's "start" class is called. Likewise, when the Wrapper sends a stop message, the main method of the application's "stop" class is called.

When the WrapperStartStopApp helper class is launched, it needs to be told about the class names of both the "start" and "stop" classes as well as any parameters that need to be provided to the main methods of each class. This results in a parameter list that is a little more complicated than that of the Method 1 (WrapperSimpleApp) helper class.

The first parameter passed to the WrapperStartStopApp class will be the full class name of the "start" class. This is followed by a count of the parameters to the "start" class's main method that will come next. After the "start" class's parameters, comes the "stop" class's full class name. This is followed by a TRUE/FALSE flag that tells the WrapperStartStopApp class whether or not it should wait until all non-daemon threads have completed before actually exiting. This flag is then followed by the "stop" class's parameter count and parameters. Don't worry if this is confusing right now. A detailed example is provided below.

Detailed Instructions

This section will walk you through a detailed explanation of how to configure Tomcat to run within the Wrapper. Most other applications can be integrated by following the same steps.

Install Tomcat

This tutorial will start with a clean install of Tomcat. We used Tomcat 9.0.0.M13 so the exact steps may be slightly different depending on the version you installed. Tomcat was installed in the /usr/lib directory, resulting in a Tomcat home directory of /usr/lib/apache-tomcat-9.0.0.M13.

Installing Wrapper Files

There are four directories that are required to be configured in order to be able to use the Wrapper.

NOTE

Please make sure that you are using the appropriate Wrapper, and libwrapper.so files that were built for the platform being run. It sounds obvious, but the Wrapper Linux version will not work on Solaris for example.

bin directory

The Wrapper is shipped with a shell script (sh) that can be used to reliably start and stop any Java application controlled by the Java Service Wrapper.

First, copy the following file into the Tomcat bin directory (on older Wrapper versions, this file was named 'sh.script.in'):

{WRAPPER_HOME}/bin/wrapper
{WRAPPER_HOME}/src/bin/App.sh.in

Rename the script file to reflect the name of the application.

{TOMCAT_HOME}/bin/tomcat

Now open the script into an editor. We need to set the long and short names to reflect that the script is being used to launch Tomcat. You will see two variables immediately after the header of the script. APP_NAME and APP_LONG_NAME. Suggested values for these variables are shown below.

APP_NAME="tomcat"
APP_LONG_NAME="Tomcat Application Server"

The script should not require any additional modification. However, it does assume that the wrapper.conf file will be located within a conf directory (one level up, ../conf/wrapper.conf). If you wish to place the wrapper.conf file somewhere else, the WRAPPER_CONF variable in the script will require appropriate modification.

NOTE

Important! Before proceeding, please make sure that all files copied into the bin directory have their executable bit set.

lib directory

Copy the following two files into the lib directory of Tomcat:

{WRAPPER_HOME}/lib/libwrapper.so
{WRAPPER_HOME}/lib/wrapper.jar

The libwrapper.so file is a native library file required by the portion of the Wrapper that runs within the JVM. The wrapper.jar file contains all of the Wrapper classes.

NOTE

Note that the native library follows slightly different naming conventions on some platforms. Possible names include; libwrapper.a, libwrapper.sl, libwrapper.so, and libwrapper.jnilib. In any case, the file should be copied over without changing the extension.

conf directory

The Wrapper requires a configuration file "wrapper.conf" for each application. The standard location for this file is in a conf directory in the application's home directory. Copy the following template file wrapper.conf.in into the conf directory of Tomcat.

{WRAPPER_HOME}/src/conf/wrapper.conf.in

Rename the file as follows. Be sure to remove the .in extension so that the file is named wrapper.conf.

You should now have:

{TOMCAT_HOME}/conf/wrapper.conf

If you wish to relocate the configuration file wrapper.conf, you are free to do so. You will need to modify the scripts copied into the bin directory above, to reflect the new location.

logs directory

The default configuration file wrapper.conf will place a wrapper.log file in a logs directory under the application's home directory. Tomcat already has such a directory, so we are all set.

{TOMCAT_HOME}/logs

If you wish to place the wrapper.log file in another location, you will need to edit the wrapper.conf file and modify the wrapper.logfile property to reflect the new location.

Locate the Application's Java Command Line

Before the Wrapper can be configured to launch an application, you will need to know the full Java command that is normally used.

Most applications make use of a script to build up the actual command line. These scripts tend to get quite unwieldy but in fact, the featured ability to avoid having to work with them is one of the benefits of working with the Wrapper.

Tomcat is launched by default using a script called startup.sh and then shutdown using a script called shutdown.sh. It is launched by first changing the current directory to the bin directory and then run from there. If you open startup.sh into an editor, you will notice after some investigation, that Java is not actually launched from this script. Rather another script, catalina.sh, is called. Tomcat's scripts are very advanced and allow the user to do a lot of configuration from the command line. The command line that we will capture and use with the Wrapper will actually be a snapshot of one such configuration. This example will assume that no parameters are passed to either the startup or shutdown scripts when they are run.

If you open catalina.sh into an editor and scroll down towards the bottom of the file, you will see a section that responds to the "start" command. There are then two options for launching the JVM with or without a Security Manager. To keep things simple, we will use the version without a security manager. The lines we are interested in look like the following:

eval $_NOHUP "\"$_RUNJAVA\"" "\"$LOGGING_CONFIG\"" $LOGGING_MANAGER $JAVA_OPTS $CATALINA_OPTS \
  -classpath "\"$CLASSPATH\"" \
  -Dcatalina.base="\"$CATALINA_BASE\"" \
  -Dcatalina.home="\"$CATALINA_HOME\"" \
  -Djava.io.tmpdir="\"$CATALINA_TMPDIR\"" \
  org.apache.catalina.startup.Bootstrap "$@" start \
  >> "$CATALINA_OUT" 2>&1 "&"

The majority of the script has the task of collecting system specific information and storing that information into environment variables. The line above then expands all of the collected information into the final Java command that launches the application.

From looking at the source of the script, we hope you appreciate the complexity and the desire to have to avoid completely writing such scripts yourself.

In order to configure the Wrapper, all that is really needed is the final expanded command line. Rather than reading through the entire script and attempting to understand it, we will use a simple trick to display the final command line in the console. Edit the script catalina.sh by changing it as follows:

#eval $_NOHUP "\"$_RUNJAVA\"" "\"$LOGGING_CONFIG\"" $LOGGING_MANAGER $JAVA_OPTS $CATALINA_OPTS \
  #-classpath "\"$CLASSPATH\"" \
  #-Dcatalina.base="\"$CATALINA_BASE\"" \
  #-Dcatalina.home="\"$CATALINA_HOME\"" \
  #-Djava.io.tmpdir="\"$CATALINA_TMPDIR\"" \
  #org.apache.catalina.startup.Bootstrap "$@" start \
  #>> "$CATALINA_OUT" 2>&1 "&"
echo "\"$_RUNJAVA\"" "\"$LOGGING_CONFIG\"" $LOGGING_MANAGER $JAVA_OPTS $CATALINA_OPTS \
  -classpath "\"$CLASSPATH\"" \
  -Dcatalina.base="\"$CATALINA_BASE\"" \
  -Dcatalina.home="\"$CATALINA_HOME\"" \
  -Djava.io.tmpdir="\"$CATALINA_TMPDIR\"" \
  org.apache.catalina.startup.Bootstrap "$@" start"

If you now rerun the script, you will see something like the following in the console (your output will all be on one line):

/opt/jdk1.8.0_45/bin/java
  "-Djava.util.logging.config.file=/usr/lib/apache-tomcat-9.0.0.M13/conf/logging.properties" 
  -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
  -Djdk.tls.ephemeralDHKeySize=2048 
  -Djava.protocol.handler.pkgs=org.apache.catalina.webresources 
  -classpath "/usr/lib/apache-tomcat-9.0.0.M13/bin/bootstrap.jar:/usr/lib/apache-tomcat-9.0.0.M13/bin/tomcat-juli.jar" 
  -Dcatalina.base="/usr/lib/apache-tomcat-9.0.0.M13" 
  -Dcatalina.home="/usr/lib/apache-tomcat-9.0.0.M13" 
  -Djava.io.tmpdir="/usr/lib/apache-tomcat-9.0.0.M13/temp" 
  org.apache.catalina.startup.Bootstrap start

We now need to repeat the same process for the stop command. Scroll down in the editor to find the "stop" command:

exec "$_RUNJAVA" $JAVA_OPTS \
  -classpath "$CLASSPATH" \
  -Dcatalina.base="$CATALINA_BASE" \
  -Dcatalina.home="$CATALINA_HOME" \
  -Djava.io.tmpdir="$CATALINA_TMPDIR" \
  org.apache.catalina.startup.Bootstrap "$@" stop

Once again, modify this section as follows:

#exec "$_RUNJAVA" $JAVA_OPTS \
  #  -classpath "$CLASSPATH" \
  #  -Dcatalina.base="$CATALINA_BASE" \
  #  -Dcatalina.home="$CATALINA_HOME" \
  #  -Djava.io.tmpdir="$CATALINA_TMPDIR" \
  #  org.apache.catalina.startup.Bootstrap "$@" stop
echo "exec $_RUNJAVA $JAVA_OPTS" \
    "-classpath $CLASSPATH" \
    "-Dcatalina.base=$CATALINA_BASE" \
    "-Dcatalina.home=$CATALINA_HOME" \
    "-Djava.io.tmpdir=$CATALINA_TMPDIR" \
    "org.apache.catalina.startup.Bootstrap $@ stop"

If you now run the script catalina.sh stop, you will see something like the following in the console (your output will all be on one line):

exec /opt/jdk1.8.0_45/bin/java
  -Djdk.tls.ephemeralDHKeySize=2048 
  -Djava.protocol.handler.pkgs=org.apache.catalina.webresources 
  -classpath "/usr/lib/apache-tomcat-9.0.0.M13/bin/bootstrap.jar:/usr/lib/apache-tomcat-9.0.0.M13/bin/tomcat-juli.jar" 
  -Dcatalina.base="/usr/lib/apache-tomcat-9.0.0.M13" 
  -Dcatalina.home="/usr/lib/apache-tomcat-9.0.0.M13"
  -Djava.io.tmpdir="/usr/lib/apache-tomcat-9.0.0.M13/temp"
  org.apache.catalina.startup.Bootstrap stop

Other than the exec at the beginning of the shutdown line, and the redirected output of the startup line, the two commands are almost identical. The only difference is the parameter passed to the main class at the end. The exec portion of the shutdown command and redirection to capture console output are not required when using the Wrapper so the rest of this example will ignore those portions of the commands.

The Wrapper will also handle the quoting of elements of the Java command line that it builds up. So it is not necessary for them to be carried over into the configuration file wrapper.conf below.

Modifying the "wrapper.conf" File

In order to be able to use the above Java command line with the Wrapper, we need to break up the command line's components into a configuration file. Open the wrapper.conf file into an editor and make the changes below.

NOTE

Where properties are mentioned below, links are provided to their descriptions. Please take the time to review the descriptions of any properties that are modified. In many cases, there are further details on their usage that are not mentioned here.

Java Executable

First, extract the Java executable and assign the location path to the wrapper.java.command property:

wrapper.java.command=/opt/jdk1.8.0_45/bin/java

Java Arguments

Most applications provide a number of parameters to the Java executable when it is launched. The Wrapper provides special properties for configuring things like memory, as well as class and library paths. These will be covered below, however any other settings are configured using the wrapper.java.additional.<n> series of properties.

The Tomcat command line has several such properties:

wrapper.java.additional.1=-Dcatalina.base=/usr/lib/apache-tomcat-9.0.0.M13
wrapper.java.additional.2=-Dcatalina.home=/usr/lib/apache-tomcat-9.0.0.M13
wrapper.java.additional.3=-Djava.io.tmpdir=/usr/lib/apache-tomcat-9.0.0.M13/temp
wrapper.java.additional.4=-Djdk.tls.ephemeralDHKeySize=2048
wrapper.java.additional.5=-Djava.protocol.handler.pkgs=org.apache.catalina.webresources
# Following properties are optional:
wrapper.java.additional.6=-Djava.util.logging.config.file=/usr/lib/apache-tomcat-9.0.0.M13/conf/logging.properties
wrapper.java.additional.7=-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager

Wrapper Jar

The Wrapper requires that its wrapper.jar be specified:

wrapper.jarfile=/usr/lib/apache-tomcat-9.0.0.M13/lib/wrapper.jar

WARNING

The wrapper.jarfile property was introduced in version 3.5.55. When using earlier Wrapper versions, it is necessary to include wrapper.jar in the classpath:

wrapper.java.classpath.1=D:\apache-tomcat-9.0.0.M13\lib\wrapper.jar

Then, the indices of next classpath elements must be adjusted so that the wrapper.java.classpath.1 property is not repeated.

Classpath

Next, comes the classpath, which is configured using the wrapper.java.classpath.<n> properties. The Wrapper requires that the classpath be broken up into its individual elements.

wrapper.java.classpath.1=/usr/lib/apache-tomcat-9.0.0.M13/bin/bootstrap.jar
wrapper.java.classpath.2=/usr/lib/apache-tomcat-9.0.0.M13/bin/tomcat-juli.jar

Main Class

The next component of the command used to launch Tomcat is the main class, org.apache.catalina.startup.Bootstrap. The main class executed by Java when launched is specified by using the wrapper.java.mainclass property. As mentioned above; however, because we are making use of the WrapperStartStopApp helper class to start and stop Tomcat, we will specify that class's full name as the main class. The Tomcat main classes are then specified as application parameters below.

wrapper.java.mainclass=org.tanukisoftware.wrapper.WrapperStartStopApp

Application Parameters

Complete details to configure these properties are available by visiting, wrapper.app.parameter.<n> Application parameters appear in the Java command line, directly after the main class.

When we use the WrapperStartStopApp helper class, a lot of information needs to be provided about both the "start" and "stop" classes. This information includes each classes' full name, the list of parameters passed to their main methods, and a flag instructing the helper class whether or not it should wait for all non-daemon threads to exit before causing the JVM to exit.

We will explain as follows, how this information is encoded. We start by presenting the property values for the Tomcat application. Several comments have been already added above, specifically about what you can normally find in the wrapper.conf file. It will be easy to understand what each property means. We suggest to add these comments to your configuration file wrapper.conf as well.

# The first application parameter is the name of the class whose main
# method is to be called when the application is launched.  The class
# name is followed by the number of parameters to be passed to its main
# method.  Then comes the actual parameters.
wrapper.app.parameter.1=org.apache.catalina.startup.Bootstrap
wrapper.app.parameter.2=1
wrapper.app.parameter.3=start

# The start parameters are followed by the name of the class whose main
# method is to be called to stop the application.  The stop class name
# is followed by a flag that controls whether or not the Wrapper should
# wait for all non daemon threads to complete before exiting the JVM.
# The flag is followed by the number of parameters to be passed to the
# stop class's main method.  Finally comes the actual parameters.
wrapper.app.parameter.4=org.apache.catalina.startup.Bootstrap
wrapper.app.parameter.5=TRUE
wrapper.app.parameter.6=1
wrapper.app.parameter.7=stop

The start and stop class names should be clear now. The first parameter count is required to locate the stop class in the parameter list. The second count is there for consistency.

The flag at parameter #5 above, is used to control the behavior of the WrapperStartStopApp helper class when it is shutting down the JVM. When the Wrapper sends a JVM shutdown request, WrapperStartStopApp responds by calling the main method of the "stop" class with the configured parameters. The flag above controls what happens when that main method returns.

If the flag is FALSE then System.exit(0) will be called immediately. When TRUE, WrapperStartStopApp will wait until all non-daemon threads have completed before calling System.exit(0). The latter is the behavior that produces the cleanest shutdown for Tomcat. If TRUE is specified, but one or more Daemon threads do not complete, the Wrapper will forcibly kill the JVM after its Shutdown Timeout has expired. The time out set by default is to 30 seconds.

Non-daemon threads are counted by iterating over all threads in the system and counting those whose isDaemon method returns FALSE. Unfortunately, this count will never actually reach "0" (zero) on most JVMs because of the existence of system threads.

In most Oracle Hotsopt JVMs, there will be one non-daemon system thread. In order to make the shutdown work correctly, this system thread count needs to be correct as well. It can be set by defining a system property: org.tanukisoftware.wrapper.WrapperStartStopApp.systemThreadCount The default value is "1 thread".

NOTE

If the main method of the stop class calls System.exit from within its main thread, that thread will in effect become deadlocked by that call. The Wrapper avoids a deadlock by detecting this and proceeding with the shutdown after 5 seconds. This may however result in the application failing to shutdown cleanly on its own and should be avoided where possible.

This case can be tested for by enabling the wrapper.debug=TRUE property and then observing the log file during the shutdown process.

Library Path

In order to use the Wrapper, one more property must be set. The Wrapper makes use of a native library to control interactions with the system. This library file libwrapper.so needs to be specified on the library path supplied to the JVM.

Tomcat does not have any native libraries of its own, but if it did, the directories where they were located would also need to be specified. The library path is set using the wrapper.java.library.path.<n> properties.

wrapper.java.library.path.1=/usr/lib/apache-tomcat-9.0.0.M13/lib

Putting It All Together

Putting it all together, we get the following:

wrapper.java.command=/opt/jdk1.8.0_45/bin/java

wrapper.java.additional.1=-Dcatalina.base=/usr/lib/apache-tomcat-9.0.0.M13
wrapper.java.additional.2=-Dcatalina.home=/usr/lib/apache-tomcat-9.0.0.M13
wrapper.java.additional.3=-Djava.io.tmpdir=/usr/lib/apache-tomcat-9.0.0.M13/temp
wrapper.java.additional.4=-Djdk.tls.ephemeralDHKeySize=2048
wrapper.java.additional.5=-Djava.protocol.handler.pkgs=org.apache.catalina.webresources
#following properties are optional:
wrapper.java.additional.6=-Djava.util.logging.config.file=/usr/lib/apache-tomcat-9.0.0.M13/conf/logging.properties
wrapper.java.additional.7=-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager

wrapper.jarfile=/usr/lib/apache-tomcat-9.0.0.M13/lib/wrapper.jar

wrapper.java.classpath.1=/usr/lib/apache-tomcat-9.0.0.M13/bin/bootstrap.jar
wrapper.java.classpath.2=/usr/lib/apache-tomcat-9.0.0.M13/bin/tomcat-juli.jar

wrapper.java.library.path.1=/usr/lib/apache-tomcat-9.0.0.M13/lib

wrapper.java.mainclass=org.tanukisoftware.wrapper.WrapperStartStopApp

wrapper.app.parameter.1=org.apache.catalina.startup.Bootstrap
wrapper.app.parameter.2=1
wrapper.app.parameter.3=start
wrapper.app.parameter.4=org.apache.catalina.startup.Bootstrap
wrapper.app.parameter.5=TRUE
wrapper.app.parameter.6=1
wrapper.app.parameter.7=stop

Notice, while these configurations will work correctly on this particular machine, it is highly dependent on the directory structure and platform. By taking advantage of the fact that the Wrapper's scripts always set the current directory to the location of the script, and by making use of a single environment variable, we are able to modify the above properties so that they are completely platform and machine independent.

wrapper.java.command=%JAVA_HOME%/bin/java

wrapper.java.additional.1=-Dcatalina.base=..
wrapper.java.additional.2=-Dcatalina.home=..
wrapper.java.additional.3=-Djava.io.tmpdir=../temp
wrapper.java.additional.4=-Djdk.tls.ephemeralDHKeySize=2048
wrapper.java.additional.5=-Djava.protocol.handler.pkgs=org.apache.catalina.webresources
#following properties are optional:
wrapper.java.additional.6=-Djava.util.logging.config.file=../conf/logging.properties
wrapper.java.additional.7=-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager

wrapper.jarfile=../lib/wrapper.jar

wrapper.java.classpath.1=../bin/bootstrap.jar
wrapper.java.classpath.2=../bin/tomcat-juli.jar

wrapper.java.library.path.1=../lib

wrapper.java.mainclass=org.tanukisoftware.wrapper.WrapperStartStopApp

wrapper.app.parameter.1=org.apache.catalina.startup.Bootstrap
wrapper.app.parameter.2=1
wrapper.app.parameter.3=start
wrapper.app.parameter.4=org.apache.catalina.startup.Bootstrap
wrapper.app.parameter.5=TRUE
wrapper.app.parameter.6=1
wrapper.app.parameter.7=stop

NOTE

It has been reported that Tomcat 5.0.28 will not work correctly if the bin directory is included in the java.endorsed.dirs system property. This is caused by a change in Tomcat, rather than any issue with the Wrapper. Please modify the above configuration as follows:

wrapper.java.additional.1=-Djava.endorsed.dirs=../common/endorsed

Trying It Out

Tomcat can now be run by simply executing the script bin/tomcat console. Because of the way the Wrapper sets its current directory, it is not necessary to run this script from within the bin directory.

As you will see if you omit a command, the scripts shipped with the Wrapper are fairly standard Daemon scripts. They accept console, start, stop, restart, and dump commands. The start, stop, and restart commands are common to most Daemon scripts and are used to control the Wrapper and its application as a Daemon process. The status command can be used to find out whether or not the Wrapper is currently running. The console command will launch the Wrapper in the current shell, making it possible to kill the application with CTRL-C. The final command, dump, will send a "kill -3" signal to the Wrapper causing the its JVM to do a full thread dump.

Congratulations. Your application should now be up and running.

If you have any problems, please take a look at the Troubleshooting section for help with tracking down the problem.

Advanced

Tuning The Startup

By default, the WrapperStartStopApp class will wait 2 seconds for the user application's main method to complete. After that, it assumes that the application has started and reports back to the Wrapper process. This is done because many user applications are written with main methods that do not return for the life of the application. In such cases, there is no reliable way for the WrapperStartStopApp class to tell when and if the application has completed its startup.

If, however, it is known that the application's main method will return once the application is started, it would be ideal for the Wrapper to wait until it has done so before continuing.

waitForStartMain System Property:

For main methods that return in this way, the WrapperStartStopApp looks for the org.tanukisoftware.wrapper.WrapperStartStopApp.waitForStartMain system property. If it is set to TRUE, the WrapperStartStopApp will wait indefinitely for the main method to complete.

Example: (enable to wait)
wrapper.java.additional.1=-Dorg.tanukisoftware.wrapper.WrapperStartStopApp.waitForStartMain=TRUE

maxStartMainWait System Property:

Waiting indefinitely is an advantageous option if it known for sure that the main method will return in a timely manner. But on the other hand, while it is waiting eternally, the Wrapper will never give up on the startup process, no matter how long it takes.

So, if there is any chance that this startup could hang, then the org.tanukisoftware.wrapper.WrapperStartStopApp.maxStartMainWait system property to set the maximum wait time may be a better option. For instance, to wait for up to 5 minutes (300 seconds) for the startup main method to complete, set the property to 300 as follows:

The default value is "2 seconds".

Example: (300 seconds)
wrapper.java.additional.1=-Dorg.tanukisoftware.wrapper.WrapperStartStopApp.maxStartMainWait=300

NOTE

The main methods of many applications are designed not to return. In these cases, you must either stick with the default 2 second startup timeout or specify a slightly longer timeout, using the maxStartMainWait property, to simulate the amount of time your application takes to start up.

WARNING

If "TRUE" in the waitForStartMain is specified for an application whose start method never returns, the Wrapper will appear at first to be functioning correctly. However, the Wrapper is actually in a wait status eternally and will never enter a running state where it can monitor your application.