Professional Documents
Culture Documents
Apache 2.X + Tomcat 4.X + Load Balancing (Or Private JVMS) : Pascal Forget Matt Raible
Apache 2.X + Tomcat 4.X + Load Balancing (Or Private JVMS) : Pascal Forget Matt Raible
x
+ Load Balancing (or Private JVMs)
January 24, 2002 by Pascal Forget
Revised September 25, 2002 by Matt Raible
Original Article at http://www.ubeans.com/tomcat/.
This article contains step by step instructions for configuring an Apache 2.x web server which handles static
content and delegates JSP (Java Server Pages) and Servlet requests to two Tomcat 4.x servers using AJP
13 connectors and a load balancing worker.
Introduction
Apache 2.0 is a standards compliant, fast and mature web server which excels at delivering static content
such as static HTML pages and images. The Tomcat web server is great for serving Java Server Pages and
servlets, but it is not as fast as Apache for delivering static content.
In order to build a fast, scalable web application, the requirements call for an Apache server that delegates
servicing of JSP and servlet requests to multiple tomcat servers by using an Apache module, mod_jk, that
performs load balancing with session affinity, also known as "sticky" sessions.
Session affinity explained. When a client browser requests a JSP page for the first time, the load balancer
redirects the request received by Apache to one of the two tomcat servers; further requests originating from
the same client session will be automatically forwarded to the same tomcat server, so that the user's
session data is retrieved.
This document describes how I configured Apache 2.x to dispatch JSP and servlet requests to two Tomcat
4.x instances listening on different ports. This setup was done on a Linux system. Your mileage may vary.
• Apache 2.x Binaries (If you plan to use ssl, get the source)
• Tomcat 4.x Binaries
• The JK module
Linux: gunzip the *.gz you downloaded, untar and run install-bindist.sh
For *nux, to install Apache 2.0.42 with mod_sll installed, you will need to compile from
source:
$ lynx http://www.apache.org/dist/httpd/httpd-2.0.42.tar.gz
$ gzip -d httpd-2.0.42.tar.gz
$ tar xvf httpd-2.0.42.tar
$ ./configure --enable-mods-shared=most --enable-ssl=shared
$ make
$ make install
Then download mod_jk-2.0.42.so and put it into your modules directory and rename it
mod_jk.so.
Windows: Execute the downloaded executable and install.
Edit the Apache server's configuration file httpd.conf which is located in the /usr/local/apache2/conf
directory.
2.2.1 Below "# LoadModule foo_module modules/mod_foo.so", insert the following lines:
#
# Load mod_jk
#
LoadModule jk_module modules/mod_jk.so
#
# Configure mod_jk
#
JkWorkersFile conf/workers.properties
JkLogFile logs/mod_jk.log
JkLogLevel info
2.2.2 Below the "DocumentRoot" line, insert the following two lines:
Now we will create a file called worker.properties, and we will place it under /usr/local/apache2/conf. The
worker.properties file tells Apache about the various Tomcat servers that are running, and on which port
they are listening.
In my setup, I installed the two Tomcat servers in different directories, on the same machine as Apache.
Feel free to put your Tomcat servers on different machines.
I made the first Tomcat server's AJP13 connector listen on port 11009 instead of the default port which is
8009, and the second one listens on port 12009.
I have decided to name my tomcat servers tomcat1 and tomcat2. This is purely my choice.
# ------------------------
# First tomcat server
# ------------------------
worker.tomcat1.port=11009
worker.tomcat1.host=localhost
worker.tomcat1.type=ajp13
# Specify the size of the open connection cache.
#worker.tomcat1.cachesize
#
# Specifies the load balance factor when used with
# a load balancing worker.
# Note:
# ----> lbfactor must be > 0
# ----> Low lbfactor means less work done by the worker.
worker.tomcat1.lbfactor=100
# ------------------------
# Second tomcat server
# ------------------------
worker.tomcat2.port=12009
worker.tomcat2.host=localhost
worker.tomcat2.type=ajp13
#
# Specifies the load balance factor when used with
# a load balancing worker.
# Note:
# ----> lbfactor must be > 0
# ----> Low lbfactor means less work done by the worker.
worker.tomcat2.lbfactor=100
# ------------------------
# Load Balancer worker
# ------------------------
#
# The loadbalancer (type lb) worker performs weighted round-robin
# load balancing with sticky sessions.
# Note:
# ----> If a worker dies, the load balancer will check its state
# once in a while. Until then all work is redirected to peer
# worker.
worker.loadbalancer.type=lb
worker.loadbalancer.balanced_workers=tomcat1, tomcat2
#
# END workers.properties
#
In both /usr/local/tomcat1 and /usr/local/tomcat2, the same files will be modified. I here by present the
modifications made to the files contained in the /usr/local/tomcat1 directory tree structure. You should also
apply the same changes to the corresponding files located under the /usr/local/tomcat2 directory tree
structure.
In my many years of consulting, I have learned not to rely on environment variables which can be unset by
ignorant or malicious people. This is why I explicitely set the JAVA_HOME and CATALINA_HOME variables
directly in the catalina.sh file.
At line 32, before the "# ----- Verify and Set Required Environment Variables " line, insert the following two
lines:
with:
<Server port="8005"
with:
<Server port="11005"
For the tomcat2 server, replace port 8005 with 12005. This will prevent the two servers from conflicting.
port="8009"
with:
port="11009"
Example:
NOTE: If you don't comment this out, you will need to change the port numbers so they don't conflict
between tomcat instances.
Example:
<Service name="Tomcat-Apache">
<!--
<Connector className="org.apache.catalina.connector.warp.WarpConnector"
port="8008" minProcessors="5" maxProcessors="75"
enableLookups="true" appBase="webapps"
acceptCount="10" debug="0"/>
-->
NOTE: You might want to comment out the entire <Service name="Tomcat-Apache"> element. If so, make
sure and remove the comments within it - XML doesn't like comments within comments.
3.3.1 Create a file named index.jsp and put it in the /usr/local/tomcat1/webapps/ROOT directory:
<html>
<body bgcolor="red">
<center>
<%= request.getSession().getId() %>
<h1>Tomcat 1</h1>
</body>
</html>
3.3.2 Create a file named index.jsp and put it in the /usr/local/tomcat2/webapps/ROOT directory:
<html>
<body bgcolor="blue">
<center>
<%= request.getSession().getId() %>
<h1>Tomcat 2</h1>
</body>
</html>
Click on: http://localhost/. You should see the default Apache index.html page.
Now test that tomcat (either Tomcat 1 or Tomcat 2) is serving Java Server Pages.
If you get a red page, the page was served by the tomcat1 server, and if you get a blue page, it was served
by the tomcat2 server.
Now test that session affinity - also known as sticky sessions - works within the load balancer. Hit the reload
button of your web browser several times and verify that the index.jsp page you get is always received from
the same tomcat server.
NameVirtualHost *
<VirtualHost *>
ServerName localhost1
JkMount /*.jsp tomcat1
JkMount /servlet/* tomcat1
</VirtualHost>
<VirtualHost *>
ServerName localhost2
JkMount /*.jsp tomcat2
JkMount /servlet/* tomcat2
</VirtualHost>
Where the serverNames are fully-qualified host names in a DNS Server. More information can be found at
http://httpd.apache.org/docs-2.0/vhosts/.
NOTE: When using SSL with multiple Virtual Hosts, you must use an ip-based configuration. This is
because SSL requires you to configure a specific port (443), whereas name-based specifies all ports (*).
You might the following error if you try to mix name-based virtual hosts with SSL.
Supplemental Information
Question 1:
Why did you choose to use the AJP13 connector rather than the WARP connector that is recommended?
Answer:
The warp connector is used in conjunction with mod_webapp, and mod_webapp does not currently support
load balancing.
Also, I found the documentation for the warp connector on the Jakarta web site to be quite lacking. See:
http://jakarta.apache.org/tomcat/tomcat-4.0-doc/config/warp.html
I know that the future lies in the warp connector, but in the meantime, I needed something. The
documentation did not explain to me exactly what benefits I would get from using the Warp connector as
opposed to AJP13.
Question 2:
You might specify that creating two instances of the tomcat installation is not needed as you can share the
main binaries and libs by specifying 2 distinct CATALINA_BASE variables.
True, but in real life the two tomcat servers are usually located on two different machines. My setup might
be overkill for a single machine setup, but it's easy to tar up the "tomcat2" server and put it on a second
machine; you just have to change "localhost" to the appropriate machine name in
/usr/local/apache2/conf/workers.properties and you're done.
Question 3:
What does not work and what does work in load balancing?
Answer:
Load balancing works great.
Some people are working on sessions that will be replicated across all tomcat servers in the cluster, so I'll
just wait for it to become available rather than make a homebrewed distributed session mechanism.
The downside of not having sessions replicated across all the tomcat servers in the cluster is that if one
tomcat server dies, all the sessions that it contained are lost, which usually makes a lot of unhappy users.
2. Failover works
If one tomcat server dies, the load balancer then "rebalances" the queries to the remaining tomcat servers.
3. Failback works
When a tomcat server comes back from the dead, the load balancer automatically starts to send queries to
it. So you can actually add capacity to your cluster on the fly.
References
For more information, you should read An Apache Load Balancing Cluster. It talks about mod_jserv, which
is now mod_jk, and it uses JServ instead of Tomcat, but the concepts are still valid.
Conclusion
The list of steps that are required to obtain a scalable web application solution based on Apache 2.x and a
group of distibuted Tomcat servers are well-defined and if you follow the receipe exactly, you should be able
to achieve success.