Eighth IEEE International Symposium on Cluster Computing and the Grid
TCP Connection Scheduler
in Single IP Address Cluster
Hajime Fujita1 , Hiroya Matsuba2 , and Yutaka Ishikawa1,2
1 2
Graduate School of Information Science and Technology,
The University of Tokyo
{hfujita@is.s, matsuba@cc, ishikawa@is.s}.u-tokyo.ac.jp
Abstract
A broadcast-based single IP cluster aims at being both
scalable and available. However, existing systems can only
employ static traffic assignment based on incoming pack-
ets. In this paper we propose FTCS, a new TCP connection
dispatching mechanism that enables a single IP cluster to
use more flexible load balancing algorithms. In this mech-
anism, one of the cluster nodes acts as a master node. A
centralized connection scheduler runs on the master node
in order to dispatch TCP connections to nodes of the clus-
ters. Since connections are scheduled by a single scheduler,
the master node is able to employ arbitrary scheduling al-
gorithms. Once a TCP connection is established on a node,
succeeding communication is handled without involving the
master node. When the master node fails, one of the nodes
takes over the role of the master node. Therefore the master
node does not become a single point of failure. Benchmark
results using SPECweb2005 Support benchmark show that
a four-node Linux cluster using FTCS balances workloads
well and successfully handles 13% more requests than the
existing method, on average.
1 Introduction
Today, as the Internet is an essential part of the infras-
tructure for society, server computers are one of the most
important components of Internet services. The increas-
ing population of Internet users, network bandwidth and
the processing capabilities of commodity level computers
require more performance from servers. Furthermore, as
servers are the key components of services, they are re-
This work has been partially supported by the CREST project of JST
(Japan Science and Technology Agency).
978-0-7695-3156-4/08 $25.00 © 2008 IEEE
DOI 10.1109/CCGRID.2008.75
Information Technology Center,
The University of Tokyo
quired to be available every day of the week, or even every
day of the year.
Constructing a server from a computer cluster [4] is one
solution to these requirements. It is desirable for a cluster
server to make only one IP address visible to clients, since
many Internet applications think that a server is constructed
from a single computer, hence a single IP address. Hiding
multiple cluster nodes behind one address is also preferable
because a cluster can dispatch a request to an arbitrary node
while clients use only one address. This makes it possible
to implement load balancing, hot node addition, or hot node
removal. There are already two approaches for constructing
single IP clusters, the centralized dispatcher approach, such
as that of Linux Virtual Server[15, 10], TCP Router[6], and
SAPS[8], and the broadcast based approach, such as that of
Windows NLB[1], ONE-IP[5], Clone Cluster[14], and Hive
Server[13].
In centralized dispatcher-based clusters, the dispatcher
has the IP address of the cluster and bridges the cluster and
its client. This type balances workloads well but has a single
point of failure at the dispatcher. In broadcast-based clus-
ters, all nodes have the same IP address so that all nodes
may communicate with clients directly. In order to guaran-
tee that only one node communicates with a client at a time,
a static load balancing mechanism is employed. This type
does not have a single point of failure, but its load balancing
capability is limited.
In this paper, we propose a new method, FTCS (Flexi-
ble TCP Connection Scheduling). FTCS employs a mas-
ter node, which dispatches new TCP connections to one of
the nodes. Another node is able to communicate with the
clients without the master after establishing a connection.
FTCS enables broadcast-based single IP clusters to employ
dynamic and flexible connection scheduling, while preserv-
ing robustness against node failures.
We have implemented FTCS in the Linux kernel. Most
366

Figure 1. An example of centralized
dispatcher-type clusters.
of the features have been implemented as a kernel module
by using the Linux network packet filtering mechanism.
We have evaluated FTCS by using the SPECweb2005
Support benchmark test. The results show that FTCS well
balances the disk I/O load among four nodes and handles
13% more requests than the existing static connection as-
signment method.
2 Background
There have been quite a few single IP cluster systems
proposed. We present a brief view of the existing systems
and divide them into two groups, the centralized dispatcher
type and the broadcast-based type.
2.1 Centralized dispatcher type
Centralized dispatcher type single IP clusters generally
have one dispatcher and multiple back-end nodes (Figure1),
examples of this type include Linux Virtual Server[15, 10],
TCP Router[6], and SAPS[8]. The dispatcher has an IP ad-
dress which is visible to clients and receives all incoming
packets from clients. When the dispatcher receives a packet,
it chooses one of the back-end nodes based on a load bal-
ancing algorithm, then forwards the packet to it. Since con-
nection dispatching is controlled by the single director, it is
able to assign a connection to an arbitrary node chosen by
the algorithm.
It is necessary for these systems to work correctly so that
all packets that belong to the same connection are forwarded
to the same back-end node. The dispatcher has a connection
tracking table in order to ensure this.
One of the drawbacks of this type is that the dispatcher is
a single point of failure for the cluster since back-end nodes
are not able to receive any packets from clients unless they
are forwarded by the dispatcher. Since the dispatcher has a
connection tracking table, it is difficult for another node to
take over the role of dispatcher without losing connections
between clients and back-end nodes. Moreover, even if a
connection tracking table can be reconfigured, every com-
munication between the server and a client, including exist-
ing connections, hangs during the dispatcher failover, since
367
Figure 2. An example of broadcast-based
clusters.
no back-end node can communicate with its client without
the dispatcher.
2.2 Broadcast-based type
Broadcast-based single IP cluster (Figure2), like Win-
dows NLB[1], ONE-IP[5], Clone Cluster[14], and Hive
Server[13], does not have a special node like the dispatcher
in the previous type. All nodes in the cluster have the same
IP address that can be accessed by clients. Each incom-
ing packet is broadcasted toward all nodes, then one of the
nodes accepts it and responds to the client. The others ig-
nore and discard the packet. This type does not have a single
point of failure since all nodes are able to receive incoming
packets without depending on another node.
It is a fundamental issue to implement a mechanism that
allows only one node to respond to the request. If no one
responds, the client thinks that the packet is lost or, in
the worst case, the server is dead. On the other hand, if
more than two nodes respond, the client is confused by the
server’s seemingly irregular behaviors.
Since it is too heavy to decide inside the cluster which
node should respond upon every packet arrival, existing sys-
tems use a static hash function to filter incoming packets.
For example, whether the node i (0 ≤ i ≤ N − 1) should
respond to the incoming packet is determined by the follow-
ing formula,
k ≡ i (modN )
where N is the number of cluster nodes, k is the hash key
constructed from a packet. For a hash key, an IP address, a
port number, or their combination is often used.
These hash functions also aim at balancing network traf-
fic among cluster nodes. The basic idea is that if sources of a
hash key (e.g. IP address or port number) are uniformly dis-
tributed, incoming traffic should be equally balanced among
cluster nodes.
However, even if the connection requests are uniformly
distributed, the workload required to process each connec-
tion request is not guaranteed to be equal. For example at
the Web server, the total workload per connection, such as
running scripts or reading files, varies among connections.
Existing broadcast-based single IP clusters can do nothing
about this kind of unbalanced load since the size of the
 1: procedure R ECEIVE TCPS EGMENT (s: Segment)
2: q: quadraple ← (srcIP, srcPort, dstIP, dstPort)
3: tcb: TCB ← findTCB(q)
4: if SYN is set and ACK is not set then
5: if tcb found and its state is TIME-WAIT then
6: remove tcb from the Connection Table
7: end if
8: if node-state is MASTER then
9: n: Node
10: if q ∈ ForwardingHistoryTable then
11: n ← history-table(q)
Figure 3. An overview of an FTCS cluster. 12: else
13: n ← schedule()
14: update-history-table(q, n)
15: end if
workload per each node is completely determined by their 16: if n is myself then
clients. Furthermore, it is difficult to implement many load 17: accept s
balancing algorithms, such as round robin or least loaded, 18: else
in this type of cluster. 19: forward s to n
20: end if
21: else
3 Design 22: if s is from the master then
23: accept s
24: else
25: discard s
In this section we design the FTCS (Flexible TCP Con-
26: end if
nection Scheduler), the proposed method. Our design goal 27: end if
is to provide a scalable single IP cluster server without a 28: else
single point of failure. By scalability, we mean that loads of 29: if tcb found then
each cluster node are balanced without any bottlenecks for 30: accept s
the management. 31: else
FTCS is a method designed to construct a single IP clus- 32: discard s
33: end if
ter based on a broadcast-based approach, but has a cen-
34: end if
tralized connection scheduler to improve load balancing of 35: end procedure
TCP server applications. FTCS introduces two types of
cluster nodes, master and slave. The master node is a con- Figure 4. Algorithm for handling incoming
nection scheduler, which assigns a new TCP connection to TCP segments.
one of the cluster nodes. All other nodes are slave nodes.
The only difference between the master and the slaves is
how they handle a new TCP connection request. Once a
node (either the master or a slave) accepts a connection
request, the node handles the connection, but other nodes
discard packets for that connection independently. This
method is focused on TCP communications, and relies on
its state control mechanism, so other stateless protocols like
UDP should be handled by the existing static assignment
method[1, 5, 14, 13].
An overview of the system is shown in Figure3. Each
node has at least two network interfaces. One is used
for communications with clients, and the other is for
intra-cluster communications. Since FTCS is based on a
broadcast-based single IP cluster, all incoming packets from
clients must be broadcasted to all the nodes. Although most
of Ethernet devices today are connected by using switches,
there are several ways to deliver packets to all nodes. This
issue is later discussed in the Section 4.

368
3.1 Incoming packet handling

The algorithm used for handling incoming TCP seg-

ments is shown in Figure4.
When a client begins a new TCP connection, it first sends
a TCP segment with the SYN bit set in its TCP header. We
call this the “SYN segment.” The master node determines
which node should handle this connection when it receives
this segment (Figure4, L.8). A connection can be accepted
by either one of the slave nodes or the master node itself.
This decision is made by a scheduling policy which can
be replaced dynamically at the run time (Figure4, L.13).
Since the connection scheduling is done by a single mas- Figure 5. Infinite ping-pong of ACKs.
ter node, it can employ many scheduling policies that are
used in dispatcher-based systems. If one of the slave nodes 3.2 Avoiding duplicated connections
is chosen, the master node forwards the SYN segment to
it (Figure4, L.19). This scheduling decision is recorded in
One of the most important issues for FTCS to implement
a table in case of retransmission, which is described later
a single IP view is how to ensure that no more than one
(Figure4, L.14). A slave node discards any SYN segments
node responds to one connection at the same time. Fail-
received directly from clients. It accepts SYN segments
ure to ensure this causes many undesirable phenomena such
only when they are forwarded from the master node. This
as packet flooding, unintentional reset of a connection, and
ensures that only one node responds to each new connection
even the destruction of transferred data.
request (Figure4, L.22). After a node accepts a SYN seg-
An example of packet flooding happens in the case that
ment, it returns a segment with the SYN and ACK bits set.
two or more nodes response to the same connection between
Then the client sends an ACK segment to the server as a re-
a client and the cluster. Such an irregular behavior may
ply to the server. This is the standard TCP 3-way handshake
occur in two cases. Before revealing the cases, the packet
procedure [11].
flooding behavior is described using an example in Figure5.
When the server transmits a SYN+ACK segment, the Suppose that the client and node 1 have an established TCP
TCP protocol stack also creates a record called a Transmis- connection. Node 2 also has been keeping the same con-
sion Control Block (TCB) to track the connection. The TCB nection information with the different state. Node 2 is , of
includes information about the two endpoints of the connec- course, not recognized by the client.
tion such as the quadruple of remote IP address, remote port A pair of two numbers like (1000, 2000) in Figure5 de-
number, local IP address, and local port number. TCB is notes (SND.NXT, RCV.NXT). SND.NXT is the sequence
stored into the TCP connection table. When a node receives number used to send a segment next time. RCV.NXT is the
a TCP segment without its SYN bit set, it consults its con- sequence number which the node will receive next. There-
nection table by using the quadruple to determine whether fore, (1000, 2000) for the client means that the sequence
the segment should be accepted (Figure4, L.3). The seg- number 1000 will be used next by the client and the peer of
ment is dropped if the node does not have a corresponding the connection (node 1) will send a segment with sequence
TCB (Figure4, L.32). This mechanism enables each node number 2000 next time. To make situations simple, let all
to determine whether it should accept a segment indepen- nodes have the same window size of 100. What happens in
dently. In other words, once a TCP connection is assigned, this case is as follows:
a slave node is able to communicate with its client without
the master. 1. The client first sends segment 1 with 100 bytes data,
which is received by both node 1 and node 2.
Unlike standard TCP implementations, FTCS does not
return an RST segment if it receives an irrelevant TCP 2. Node 1 accepts this segment and returns ACK (seg-
segment, since another node might accept it. Even if ment 3).
none of the nodes has a relevant TCB, no one returns
an RST. Though this behavior violates the original TCP 3. However, for node 2, the sequence number recorded in
specification[11], it is not fatal to most applications because segment 1 is out of its receive window (in this case,
these kinds of RST may be also masked by firewalls, even 500 to 599). TCP requires the return of ACK when a
in regular environments. node receives out-of-window segments[11]. So node

369
 2 returns ACK (segment 2). This segment has a se-
quence number from SND.NXT of node 2, which is
3000.
4. The client receives segment 2, which also overruns the
receive window. The client’s receive window is from
2000 to 2099 at this time.
5. Therefore the client returns ACK (segment 4). This
segment reaches not only node 2 but also node 1. Node
1 just ignores this segment as a duplicate. For node 2,
this is an out-of-window segment again.
6. Repeat step 3 to 5 forever. (segment 5, 6, ...)
This ping-pong of ACK segments will consume a large
amount of CPU time and network capacity. While estab-
lishing a new connection, there are two cases that can cause
this kind of irregular state.
3.2.1 Retransmission of SYN segments
The first case occurs when a packet is lost at a certain tim-
ing. When a server node accepts a SYN segment from
a client, one of the server nodes (say node 1) returns a
SYN+ACK segment to the client. If this segment is lost,
the client retransmits the initial SYN segment to the server.
Without any history information, the master node may for-
ward this retransmitted SYN segment to another node, say
node 2, because the master does not know that it is a re-
transmitted one. As a result, both nodes 1 and 2 have their
own TCB with a state of SYN-RECEIVED, at the same
time. To prevent this, retransmitted SYN segments must
be processed at the same node that initially accepts the first
SYN segment. Therefore the master node should remember
the destination of each SYN segment so that it can forward
retransmitted segments to the appropriate destination (Fig-
ure4, L.10).
3.2.2 Connections of the TIME-WAIT state
The second case occurs when an existing connection is go-
ing to be closed. Suppose that node 1 initiates the connec-
tion closing procedure. It sends a FIN segment to the client
and eventually enters the TIME-WAIT state. A problematic
situation occurs when the client reuses the same port num-
ber immediately to reconnect to the server, and the schedul-
ing policy chooses a node other than node 1 (say node 2). In
this case, node 2 establishes a new TCP connection with the
client, while node 1 still thinks that it has a connection of
the TIME-WAIT state. Once this happens, the situation de-
scribed in Figure5 occurs. To prevent this, the TIME-WAIT
socket must be removed when a new SYN segment arrives
at the cluster (Figure4, L.6).
370
3.3 Node failure
As FTCS is based on broadcast-based systems, influ-
ences of a node failure are limited to a narrow area. There
are two types of node failures, slave node crash and master
node crash.
In the case of slave node crash, the TCP connections
which belong to that node are lost. However it does not
affect other nodes’ connections. The master node should
watch its slaves. Once one of them appears to be crashed,
the master stops assigning new connections to it.
In the case of master node crash, existing connections
on the slave nodes are not affected since slaves receive in-
coming packets without the master node. The difference
between master node crash and slave node crash is that the
cluster is not able to accept a new connection after the mas-
ter node crashes. To deal with this case, the slaves should
monitor the master. If one of the slaves notices that the
master is dead, it begins an election to choose a new mas-
ter node. Since all nodes are receiving the same incoming
packets at the same time, a slave node can easily become
the master node.
All state information stored in the master node should
be designed with consideration of a node crash. The mas-
ter node should not hold states which are indispensable
for communications of other nodes. However, it may hold
states that are not necessary for communications when they
are lost. For example, a scheduling module can hold unre-
producible states such as the next node number in the round
robin scheduling module or the connection tracking table
in the least connection scheduling module. This is because
losing data for scheduling may make load balancing worse,
but does not affect communications between clients and the
server.
When a slave node becomes a new master, the SYN for-
warding history table described in Section 3.2.1 is also lost.
So the new master should collect information about half-
opened connections, which are in the SYN-RECEIVED
state, from surviving slaves to reproduce the forwarding ta-
ble. Another option is to request all other surviving slave
nodes to delete all connections in SYN-RECEIVED state.
4 Implementation
We have implemented FTCS in the Linux kernel version
2.6.20. Currently our implementation covers the TCP dis-
patching mechanism. Cluster management features such as
node failure detection and recovering are left as future work.
Since it is implemented inside the kernel, there is no need
to modify user mode applications to utilize FTCS. FTCS
also does not modify the TCP protocol, so any existing
TCP implementation is able to communicate with an FTCS
server.

Figure 6. Packet filtering inside the kernel.
4.1 Packet manipulation
Most of the features of the FTCS kernel has been imple-
mented as a network packet filter module, as Linux Virtual
Server does (Figure6). Linux provides a mechanism named
netfilter to insert packet filters dynamically. Currently we
apply a very small patch to the kernel code improve the per-
formance a little.
There are two types of filtering, one for SYN segments
and one for other TCP segments. For SYN segments, fil-
tering and forwarding is done between the IP layer and the
TCP layer. When the master node forwards the SYN seg-
ment, it uses the IP protocol type 253 to show that packet
has been forwarded by the master. When a slave node re-
ceives this packet, it just passes the payload of the packet to
the TCP protocol stack.
For TCP segments other than SYN segments, the filter-
ing is done between the Ethernet layer and the IP layer in or-
der to drop irrelevant packets as early as possible. We have
applied a small patch to the kernel here because we want
to insert our packet filtering code before the IP checksum
is calculated, which is impossible with the existing packet
filtering mechanism.
If a packet header is corrupted and the filter accepts the
packet, it will eventually be discarded because of its bad
checksum. So we skip calculating checksums in the filtering
functions in order to reduce CPU consumptions.
4.2 Broadcasting Ethernet frames
Ethernet was originally designed to employ
broadcasting-style communications. However, almost
all Ethernet devices today are connected by switches,
so broadcasting and multicasting are rarely used. To
implement broadcast-based clusters, packets arriving at the
cluster must be broadcasted to all nodes. Many methods
to deliver the packets, even when nodes are connected via
switches, have been proposed in the literature[1, 5, 14, 13].
We choose the unicast-based method described in [14].
371
We set the same unicast MAC address for each Ethernet
card on each node so that every node receives incoming
Ethernet frames without using promiscuous mode. We also
modify the FTCS kernel to mask the source MAC address
from outgoing Ethernet frames in order to avoid a switch
from learning the MAC address, as described in [1]. By us-
ing this masking, all incoming Ethernet frames are always
broadcasted by the switch because it never knows which
port the destination device is connected to.
4.3 Least connection scheduler
We have implemented a least connection scheduler as an
example of connection schedulers. While using this sched-
uler, the master node has a connection tracking table to hold
the number of connections per each node. For each new
connection request, the master searches the table for a node
with minimum connections, then increments the number for
the chosen one. The number is decreased when the master
knows the connection no longer exists.
The master node can monitor all incoming traffic to the
cluster, but it cannot see any outgoing packets from the slave
nodes. This makes it difficult for the master node to know
the exact number of connections for each node. For exam-
ple, when the master sees a FIN or an RST segment from a
client, the master thinks that a connection corresponding to
the segment is going to close. However if a slave node ab-
normally finishes a connection by sending an RST segment,
the master never knows it. Therefore we added an explicit
notification of connection closing. When a slave node trans-
mits a FIN or an RST segment, it notifies the master node
that the connection is closing. This mechanism has been
implemented as an out-bound packet filter and also imple-
mented as a kernel module.
5 Evaluation
In this section we evaluate FTCS by using the
SPECweb2005 benchmark[12] as a sample of real world
applications.
The SPECweb2005 benchmark consists of three tests,
Banking, e-Commerce, and Support. We use only the Sup-
port benchmark because the other two benchmarks hold
session information on the server side and are not suitable
for running on multiple nodes with TCP-level load balanc-
ing [8].
The Support benchmark simulates a support file distribu-
tion site for many products. Many users concurrently search
for and download support files. Requested materials are di-
vided into two groups, dynamic pages and large download
files. Dynamic pages are generated by PHP scripts and con-
tain several small images. After retrieving the page, a client
requests these images to the server. Large download files
 Table 1. Computers and software used in ex-
periments.

Server / BeSim
CPU AMD Opteron 1124 (2.2GHz×2)
Memory 2GB
HDD HITACHI HDS7225SCSUN250G
(250GB SATA)
Ethernet Device Intel Pro/1000 Server
(For communications with clients) Figure 7. The testing environment for
Broadcom NetXtreme BCM5715 (On-board) SPECweb.
(For intra-cluster communications)
Kernel Linux 2.6.20 (x86 64)
Web Server Apache 2.2.3
PHP PHP 5.1.6 size of the main memory. The HTTP keep-alive mechanism
Client is enabled and its timeout is set to one second.
CPU Intel Xeon 2.80GHz × 2 In order to reduce the CPU consumption, the Intel
Memory 1GB Pro/1000 Ethernet cards on server nodes are configured
Ethernet Device Intel Pro/1000 Server (On-board) suitable for servers. That is, two parameters, Interrupt-
Kernel Linux 2.6.18 (i386) ThrottleRate and RxDescriptors, are set to 4000 and 768,
Java VM Sun Java 1.5.0 12 respectively. This limits the maximum interrupt rate to 4000
Switches times per second.
Switch A Unknown GbE Switch
Switch B Cisco Catalyst 3750 24-T-S
Switch C Alaxala AX3630S-48T2XW
5.2 Server clustering methods

The following server clustering methods are tested and

represent support materials like device drivers or firmware
updates, the sizes of which vary from about 100Kbytes to
36Mbytes.
5.1 Common configurations
Specifications of the hardware and software used in our
experiments are shown in Table 1 and the testing environ-
ment is shown in Figure7. Four server nodes and ten client
nodes are used. Server nodes and clients are connected
with a 1Gbps Ethernet link. We defined a dedicated VLAN
for the server’s client-side interfaces so that broadcasting
frames does not affect to other irrelevant Ethernet ports.
Because of this, the server and the clients are placed in
the different IP subnets. Routing between these two sub-
nets is done in the Catalyst switch, showed as Switch B
in Figure7. BeSim is a back-end simulator defined by the
SPECweb2005 benchmark to simulate a back-end database
server. The same hardware as that of a server node is used
for BeSim.
Apache HTTP server[2] is used as a web server. It is
configured to employ the prefork multiprocessing module.
This is required when we use a PHP module. Under this
configuration, one Apache process serves one connection.
The maximum number of processes, that is, the maximum
number of concurrent connections, is set to 768 due to the
compared.
FTCS: The first configuration is a cluster using the pro-
posed method. The master node uses least connection
scheduling. This configuration is later referred to as FTCS.
PortHash: The second configuration is a broadcast-based
cluster implemented by us, in order to simulate the existing
system like Windows NLB[1]. In this configuration, each
node independently determines whether it should accept a
SYN segment based on a hash key, which is calculated us-
ing the source port number of the segment. This configu-
ration is used to compare the existing static connection as-
signment with FTCS, and referred to as PortHash.
LVS-DR: The third configuration is a dispatcher-based
cluster using Linux Virtual Server (LVS). This feature is al-
ready implemented in the standard Linux 2.6 kernels. The
LVS-DR (Direct Routing) configuration is used. In this con-
figuration, incoming packets to the cluster are first received
by the dispatcher and then forwarded to a back-end server,
but outgoing packets from back-end servers are sent directly
to the clients. Usually LVS is configured by using a dedi-
cated computer for the dispatcher. However in this bench-
mark, the dispatcher also acts as a back-end server to use
the same computing resources as those of the FTCS config-
uration. As in FTCS, least connection scheduling is chosen
for the connection scheduling algorithm. TCP connection
timer is set to 30 seconds. This configuration is referred to
as LVS-DR.

372
 Table 2. Results of SPECweb benchmark tests (Simultaneous Sessions = 2300).
Clustering File Total QoS QoS QoS Error
Method Type Requests Good Tolerable Fail
Scripts 363,614 328,142 361,449 2,165
FTCS Download 26,459 22,940 23,349 3,110 0
Total 390,073 351,082 384,797 5,275
Scripts 321,165 (0.883)1 289,686 (0.883) 311,472 (0.862) 9,692 (4.476)
PortHash Download 23,324 (0.882) 17,200 (0.750) 17,431 (0.747) 5,892 (1.895) 5.33
Total 344,488 (0.883) 306,887 (0.874) 328,904 (0.855) 15,585 (2.954)
Scripts 361,383 (0.994) 335,347 (1.022) 360,245 (0.997) 1,138 (0.526)
LVS-DR Download 26,256 (0.992) 21,615 (0.942) 21,918 (0.939) 4,337 (1.395) 0
Total 387,639 (0.994) 356,962 (1.017) 382,163 (0.993) 5,476 (1.038)
1
Values in parentheses are ratio against the corresponding values of the result of FTCS.

5.3 Results PortHash

200

Request Rate (requests/s)

Node 0
We recorded each benchmark iteration three times for Node 1
150 Node 2
each clustering method. Table 2 shows average values of Node 3
three iterations. Simultaneous Sessions represents the to- 100
tal number of clients users. Total Requests are the total
50
number of successfully completed requests. QoS is de-
fined as follows. For each request for pages generated by 0
dynamic scripts, Good means complete responses, includ- 0 500 1000 1500 2000 2500
ing an HTML page and all images, within three seconds, Time (s)

and Tolerable means responses within five seconds. Oth-

erwise the service is rated as Fail. On the other hand, for
large downloads, Good and Tolerable stands for more than
99Kbytes/s and 95Kbytes/s of download rates, respectively.
In Table 2, the values for QoS Tolerable includes the num-
ber of requests that is also counted in QoS Good. Error is
the number of unsuccessful requests due to heavy load.
Note that these results are not compliant with the
SPECweb2005 regulations, because we rebooted all nodes
before running each iteration, while the regulation requires
us to run three iterations at once. Thus these results are
not comparable to other officially published SPECweb2005
scores. However they are sufficient to reveal the difference
among server clustering methods.
According to the Total Requests column of Table 2,
FTCS handles about 13% more requests than the PortHash
method. Furthermore, the PortHash configuration often
fails to complete the benchmark test because of heavy load.
We had to try seven iterations in total to obtain three suc-
cessful results with the PortHash configuration, while the
FTCS and the LVS-DR did not fail. The PortHash method
also recorded 16 errors in the third successful run due to the
heavy load. LVS-DR handled almost the same requests as
FTCS did.
Figs. 8, 9, and 10 show more precise system activities.
These data are extracted from the third iteration for each
clustering method. These graphs include warming-up phase
Figure 8. Connection request rate with the
PortHash method.
and cooling-down phase. 1800 seconds of benchmark phase
begins at 480 seconds and ends at 2280 secconds.
Figure8 shows the rate of incoming connection requests
per each node with the PortHash method. As shown in the
figure, incoming connections are well balanced among four
nodes even when we use a static hash function. However,
Figure9 shows that the PortHash method failed to balance
disk I/O loads and both Node 0 and Node 2 was overloaded,
while FTCS balanced I/O loads well. This can be explained
as follows. While connection arriving rates are nearly equal
among four nodes, workload per each connection varies.
For example, the size of the download files differs among
connections. Therefore, once one node is overloaded, the
number of unprocessed requests grows. This suggests that
a dynamic feedback mechanism against workload increases
is needed for load balancing. Balancing disk I/O is impor-
tant for this benchmark because a huge amount of files are
read and transferred during the run, and disk I/O becomes
one of the bottlenecks of the system.
Figure10 shows the number of established TCP connec-
tions. With the PortHash method, the number of connec-
tions of Node 0 is eventually saturated around 700. This is

373
 FTCS PortHash LVS-DR
800 800 800
700 Node 0 700 Node 0 700 Node 0
Node 1 Node 1 Node 1
# of Processes

600 Node 2 600 Node 2 600 Node 2

500 Node 3 (master) 500 Node 3 500 Node 3 (dispatcher)
400 400 400
300 300 300
200 200 200
100 100 100
0 0 0
0 500 1000 1500 2000 2500 0 500 1000 1500 2000 2500 0 500 1000 1500 2000 2500
Time (s) Time (s) Time (s)

Figure 9. Total number of processes waiting for disk I/O.

FTCS PortHash LVS-DR

# of Established Connections

800 800 800

700 Node 0 700 Node 0 700 Node 0
Node 1 Node 1 Node 1
600 Node 2 600 Node 2 600 Node 2
500 Node 3 (master) 500 Node 3 500 Node 3 (dispatcher)
400 400 400
300 300 300
200 200 200
100 100 100
0 0 0
0 500 1000 1500 2000 2500 0 500 1000 1500 2000 2500 0 500 1000 1500 2000 2500
Time (s) Time (s) Time (s)

Figure 10. Total number of established TCP connections.

because the number of Apache processes is limited to 768.
Since disk I/Os are invoked from HTTP requests, waiting
for a disk input means that one TCP connection is kept es-
tablished and idle. Therefore it is reasonable to use the num-
ber of established connections as an indicator of the amount
of loading.
Figs. 9 and 10 also show that LVS-DR did not balance
workloads as well as FTCS did. This is because the dis-
patcher in LVS-DR configuration is sometimes not able to
know that a back-end node closes a connection, since it does
not see outgoing packets from back-end nodes. Especially
in the HTTP server with the keep-alive feature enabled, the
server side closes the connection after the keep-alive time-
out. By this time, the HTTP server process has already
closed the socket and is ready for the next connection. How-
ever the dispatcher thinks that there is still an established
connection. This makes load balancing worse and eventu-
ally causes some nodes to become overloaded.
6 Related Work
Hive server[13] is a system which aims at improving
load balancing in broadcast-based single IP clusters. It uses
a table to determine whether a node should accept an incom-
ing packet. When an IP packet arrives at one node, it calcu-
lates a hash key from a combination of the IP address and
the port number. The table tells whether the node should
accept the packet based on the hash key. Therefore, this
system is a kind of weighted static hash function approach.
This table is shared among all nodes, and updated peri-
odically so that idle nodes accept more packets than busy
nodes. However, this approach requires more time than our
approach to respond to an imbalance of workload.
Saru[7] supports a configuration called Active-Active.
This configuration introduces multiple dispatchers into a
Linux Virtual Server-based cluster. In this configuration,
all dispatchers are active at the same time, and receive all
incoming packets to the cluster by using broadcast or mul-
ticast, as broadcast-based clusters do. Each dispatcher em-
ploys a hash function to determine which packet to accept.
This mechanism reduces the impact of the dispatcher fail-
ure. Dispatchers exchange multicast UDP packets to inform
the list of current connections in order to keep the existing
connections alive (Connection Synchronization). However,
this mechanism consumes CPU resource and network band-
width. Moreover, it does not guarantee that all connection
to be synchronized since UDP packets may be dropped.
SAPS[8] is a mechanism for dispatcher-based clusters
to offload the TCP protocol handling to the dispatcher. It
is mainly focused on the network performance and uses
Myrinet[9] for intra-cluster network. Because congestion
control and flow control are done at the network card, back-
end nodes do not encounter packet losses due to congestions
inside the cluster. However, because the TCP protocol stack

374
completely runs on the single dispatcher, named I/O Server,
it is difficult to keep a cluster running when the I/O Server
fails.
Round Robin DNS[3] (RR-DNS) ties multiple IP ad-
dresses with one domain name. Upon request, the DNS
server chooses one of the IP addresses in a round robin
manner. This kind of DNS-based mechanism can be used
together with single IP clusters like FTCS. RR-DNS dis-
tributes requests among geographically distributed sites,
whereas single IP approaches are able to deal with load im-
balance and node failures to a fine degree.
7 Conclusion and future work
In this paper we have proposed and implemented FTCS,
a mechanism for enabling flexible load balancing of TCP
applications in broadcast-based single IP clusters. FTCS
introduces a master node as a centralized connection sched-
uler to improve load balancing. Once a TCP connection
is established, the master node is no longer involved in the
communication. This makes it possible for other nodes to
keep connections when the master node fails.
We run the SPECweb2005 Support benchmark test
against a four-node cluster. Results from the benchmark
tests show that FTCS with a least-connection scheduling al-
gorithm equally balances disk I/O loads among four nodes
and handles about 13% more requests than the existing
static connection assignment method, on average. The re-
sults also show that FTCS performs as well as Linux Virtual
Server, the existing dispatcher-type cluster implementation.
The CPU utilization rate increased by receiving and dis-
carding irrelevant packets was less than 3% during the
SPECweb2005 benchmark test, and less than 5% during
receiving TCP burst packets at 1Gbps. This indicates that
the cost for receiving all incoming packets is negligible for
most TCP server applications, and thus broadcast-based ap-
proach is a reasonable way to build single IP clusters.
As future work, we will implement cluster management
features, such as node failure detection, master node fail-
over, and hot node addition/removal. We will also evalu-
ate the downtime under node failures and compare it with a
highly available cluster based on Linux Virtual Server.
References
[1] Network Load Balancing Technical Overview.
http://www.microsoft.com/technet/
prodtechnol/windows2000serv/deploy/
confeat/nlbovw.mspx.
[2] The Apache HTTP Server. http://httpd.apache.
org/.
[3] T. Brisco. DNS Support for Load Balancing. RFC 1794
(Informational), Apr. 1995.
375
[4] V. Cardellini, E. Casalicchio, M. Colajanni, and P. S. Yu.
The state of the art in locally distributed web-server systems.
ACM Computing Surveys, 34(2):263–311, 2002.
[5] O. P. Damani, P. E. Chung, Y. Huang, C. Kintala, and Y.-M.
Wang. ONE-IP: techniques for hosting a service on a cluster
of machines. In Selected papers from the sixth international
conference on World Wide Web, pages 1019–1027, Essex,
UK, 1997. Elsevier Science Publishers, Ltd.
[6] D. M. Dias, W. Kish, R. Mukherjee, and R. Tewari. A scal-
able and highly available web server. In COMPCON ’96:
Proceedings of the 41st IEEE International Computer Con-
ference, pages 85–92, Washington, DC, USA, 1996. IEEE
Computer Society.
[7] S. Horman. Active-Active Servers and Connection Syn-
chronisation for LVS. linux.conf.au 2004 (LCA 2004), Jan.
2004.
[8] H. Matsuba and Y. Ishikawa. Single IP address cluster
for internet servers. In Proceedings of 21st IEEE Inter-
national Parallel and Distributed Processing Symposium
(IPDPS2007), 2007.
[9] Myrinet. http://www.myri.com.
[10] P. O’Rourke and M. Keefe. Performance Evaluation of
Linux Virtual Server. LISA 2001 15th Systems Administra-
tion Conference, 2001.
[11] J. Postel. Transmission Control Protocol. RFC 793, Sept.
1981.
[12] SPECweb2005. http://www.spec.org/web2005/.
[13] T. Takigahira. Hive server: high reliable cluster web server
based on request multicasting. In Proceedings of The Third
International Conference on Parallel and Distributed Com-
puting, Applications and Technologies (PDCAT’02), pages
289–294, 2002.
[14] S. Vaidya and K. J. Christensen. A single system image
server cluster using duplicated MAC and IP addresses. In
Proceedings of the 26th Annual IEEE Conference on Local
Computer Networks, pages 206–214, 2001.
[15] W. Zhang. Linux Virtual Servers for Scalable Network Ser-
vices. Linux Symposium, 2000.