Byzantine Generals Problem in the Light of P2P
Computing
Natalya Fedotova
University of Parma
Department of Information Engineering
Parma, Italy
e-mail: fedotovagtlc.unipr.it
Abstract - Security problems in P2P environment become more
complicated with the growing number of P2P networks and
applications. Hence, it is very important to find reliable
mechanisms of detection and elimination of various threats,
attack sources, and malicious nodes. In this paper we consider
how it is possible to apply some classical solutions for Byzantine
Generals Problem (BGP), originally described by Lamport and
Shostak, in a P2P environment to solve such problems.
Keywords-component: P2P, DHT networks, security, attacks,
Byzantine agreement, traitors, malicious node
I. INTRODUCTION
The number of peer-to-peer network users grows
exponentially every day. Today P2P networks can offer us the
possibility of sharing different resources and services such as
distributed (collaborative) calculations, instant messaging,
pervasive devices communicating, CPU and storage resources,
etc. With the growing number of P2P networks and
applications many types of problems appear, and the most
important of them is security.
As other networks, peer-to-peer networks are subject of
many types of threats and attacks. The list of attacks that very
often take place in P2P networks includes:.lI
* poisoning, consisting of providing resources with content,
which is different from its description,
*denial of service,
* defectioncreatedbypeers thatusethenetworkwithout
contributing resources,
* viruses, i.e. some peers can contribute for sharing
infected resources,
* malware the us o are that may contain
spyware or other malicious programs,
* filtering, i.e. preventing some resources from being
carried on the network by network operators,
* masquerading,
* spamming.
O-7803-9792-4/06/$20.OO ©2006 IEEE
Luca Veltri
University of Parma
Department of Information Engineering
Parma, Italy
e-mail: luca.veltrigunipr.it
Analyzing this list of possible threats and attacks we can
note that some of them (such as viruses, malware and
defection attacks) can take place in all types of P2P networks,
while others are exploitable only with specific P2P networks
with a certain type of architecture (centralized or
decenized) or resource-sering a ndesource-sharing
mechanisms. So, for instance, it is easy to understand that
threats such as denial of service or filtering are more likely to
happen in P2P networks with a centralized architecture based
on the principles of client-server technology. It's clear, as in
this type of networks there is a single point of failure - a
central server, who represents a visible target for legal attacks.
In the case of decentralized P2P networks (or pure P2P
networks) we can avoid these kinds of problems but meet
others.
P2P networks based on the principles of Kademlia
protocol [1], or any P2P system that uses mechanisms of
Distributed Hash Tables (DHT) can be more subject of some
other specific attacks, including: routing attacks, storage and
retrieval attacks, inconsistent behavior of some nodes [2].
In our paer we consider the possibility of application of
some solutions developed for distributed computer systems to
cope with some types of such attacks. More precisely, we try
to apply Byzantine generals problem (BGP) conception and its
mathematical model to solve some security problems of P2P
systems based on the principles of DHT.
The paper is structured as follows: section 2 describes the
main types of attacks taking place in DHT based-P2P
networks; section 3 formulates the problem of Byzantine
generals agreement and describes a brief history of its
investigation; in section 4 we explain our vision of possible
applications of some approaches and solutions of BGP in P2P
environment and give an example of how these approaches
work in the case of incorrect routing updates; section 5
contains conclusions.
. MH YE FATCSH H AE-2
ENVIRONMENT
A particular class of attacks that take place in P2P systems
using DHT-mechanisms are those that involve routing
mechanisms (routing attacks), consisting in incorrect lookup
routing, incorrect routing updates or partition. Let's consider
them with some more details.
Incorrect lookup routing takes place when some
malicious node (peer) in the network try to forward lookup
process to an incorrect or non-existent node. As we know, the
routing and the process of search in the networks of this type
is implemented by defining the closest node to the key
corresponding to the desired resource. And each step of this
search is supposed to get closer to the node responsible for a
key. But a malicious peer can confuse the process of routing
claiming that some (random) node is the closest to a key. So,
lookup process can be directed incorrectly and this can prevent
the pair key/value from being found. This kind of attack is
preventable if the requesting peer (node) is able to observe
lookup progress (step by step), i.e. to control if at the last step
the lookup has reached a node which is closer to the node
responsible for a key than a predecessor node. If it is not so,
the requesting node can resolve this problem by backtracking
to the last "well behaving" node and asking for an alternative
step that offers less progress. This checking procedure makes
...
. . processes . . ... ..
the routing and lookup
morseiu seune of thsatak
slower, but helps to avoid
Incorrect routing updates attack is caused by
malicious peer that distorts the routing tables of other peers
by sending them incorrect updates. It is possible because in
P2P networks using mechanisms of DHT peers create their
routing tables by consulting each other. As result, "well
behaving" peers direct their queries to inappropriate or non-
existent nodes. Some solutions of this problem were proposed
by Pastry system and Tarzan anonymizing network creators
[2].
The problem of partition appears while a new peer
implements the process of bootstrapping, i.e. when a new peer
contacts some existing peer to join the network. So, in the case
of contact with malicious node, this new peer can be
partitioned into an incorrect (parallel) network, which was
created by a set of malicious nodes. The same can happen if
one of the malicious nodes is cross-registered in the real
network, being able to make new nodes to be connected to the
parallel network even in the case of contact with a legitimate
node at the beginning of bootstrapping process. To resolve this
problem it is proposed to use mechanisms of trust in
bootstrapping process.
Rapid joins and leaves can be used by malicious
nodes in order to start rebalancing process on the network
unnecessarily causing excess data transfers.
Inconsistent behaviour consists in correct behaviour
of a malicious node in respect of some nodes (for instance,
nodes near the malicious node in identifier space) and poor
behaviour in respect of distant nodes. So, near nodes don't
remove this malicious node from their routing tables. This
problem can be resolved by implementing routing only
through close nodes, but almost all routing systems use hops
in the identifier space to reach the desired key speedily.
Some malicious nodes can also follow the rules of
lookup protocol correctly, but deny the existence of resources
it declared to be responsible for or refuse to serve it to users of
the network. In this case we deal with storage and retrieval
attack.
Most of these attacks could be prevented or their
effects could be reduced by introducing and using a form of
distributed trust model. In the next section we describe the
Byzantine generals problem, and later we'll see how the
solutions of this problem can be applied to cope with some
kinds of P2P attacks.
III. BYZANTINE GENERALS AGREEMENT. THE
FORMULATION AND THE HISTORY OF THE PROBLEM
Byzantine failure is an arbitrary fault that occurs during the
execution of some algorithm by a distributed computer
system. This failure is caused by Byzantine Generals' Problem
(BGP) that takes place in distributed computer systems in the
presence of malfunctioning components that give conflicting
information to other parts of the system.
.
~~~~~Orgnmally it iS an agreement problem between
g By,n
generals of the Byzantine army. In its simplest form, they must
decide unanimously one dilemma in the style of Hamlet - to
attack or not to attack some enemy army? Geographically
these generals are separated and they have to communicate
with each other through messengers. But this situation is also
complicated by the presence of traitors among the generals.
These traitors try to confuse loyal generals, sending them a
false information about decisions of other generals. So, if
general A decided to attack, a traitor (general B) sends the
message to general C in which informs him, that general A
intends to retreat and that he has the same intention. But at the
same moment, sends to A a completely different information,
e.g. informs A about his intention to attack. So, traitorous
generals gives out false information, and moreover, they do
this selectively. Thus, it becomes very difficult for loyal
generals to agree on a common decision.
A Byzantine failure can cause in computer networks such
problems as:
* a failure to pass to the next step in the algorithm;
* system's inability to correctly implement the actual
algorithm;
* arbitrary execution of a step different from one predicated
by the algorithm (incorrect hops).
This problem was originally described by L. Lamport,
R. Shostak, and M. Pease in 1982 [3]. They applied the
concept of BGP to distributed computer systems and presented
several solutions and algorithms. So, it was proved that, it is
possible to reach a fault tolerance only if less than one-third of
the generals (network components) are traitors (malicious
nodes). The possibility of this fault tolerance is defined by a
resilience t, the number of faulty processes which a system
can withstand. Thus, the resilience should be so, that t<n/3,
where n is the number of processes in the system.
 It means that to cope with t traitors there must be at possibility of such adaptations and application of the existing
least 3t +1 generals (system components or nodes) and, given solutions for some types of attacks taking place on P2P
the possibility for all generals to exchange the messages with networks.
each other, must exist some recursive (round-by-round) First of all, we consider how one solution for BGP works in
algorithm (protocol) of information exchange, equal for all the case of a small number of nodes. Lets consider a situation
generals, so, that: with incorrect routing updates in a P2P network based on DHT
1) all generals make thefinal decisions; principles for n = 4 nodes and t = 1 traitors.
2) all loyal generals decide upon the same plan of To update the information about resource keys and the
action identifiers of responsible nodes in their routing tables, the
nodes should exchange information they have with each other.
3) this final plan of loyal generals must coincide with So, each node sends to others the list of keys of the resources
final decision of one loyal general at least. it is responsible for. The loyal peers send the real information
So, the problem can be solved when correctly functioning to all the other nodes, and the traitors may send different
copnnswill be able to reach the same group agreement information to each node. Let's say, the first node (peer) sends
* components.components.
regardless of conflicting A it* was showed
And 1 1 that.
1 to others the list of keys in the form of vector NI, consisting of
some binary numbers of a fixed length that present resource
to reach this agreement the number of rounds executed on the
network must be linear in the total number of participating indentificators. The second node sends the vector N2, the third
(nodes). In 1998 J. Garay and Y. Moses presented sends three different vectors X, Y and Z to node 1, node 2 and
mpone ts
a poyoiltm procl 1 *grem
for rachn Byatn agreemen node 3, the fourth node sends the vector N4 to all. After that
theyy exchangeg the received information with each other. And
in t + 1 rounds [4]. It was proved that it is the shortest
algorithm, i.e. t ± 1 is a minimal necessary number of rounds. the traitor sends arbitrary values to all again. Hence, after this
information exchange, each peer can form its matrix. An
Also some algorithms which help to avoid such a long element nij of each matrix presents a vector that nodei
procedure were presented by Ben-Or and Rabin [5]. Their received from node j. So, in our case we have the following
protocols use the principles of randomness and significantly matrixes:
reduce the number of necessary rounds in the recursive
algorithm. Subsequently, Lewis and Saia presented the protocol
based on the principles of randomness but using random NI N2 X N4 NI N2 X N4
sampling, i.e., as the authors explain, in each round, each
processor takes input from a small random sample of all the PI = NI N2 Y N4 P2 NI N2 Y N4
other processors in distinction from Rabin's protocol, where in
each round each processor takes input from all other processors A B C D E F G H
[6]. This algorithm was created for those types of distributed
systems where the direct communication of each network's NI N2 Z N4 NI N2 Z N4
component with others is not possible because of their quantity.
The best example of such systems is a P2P network. In the next
section we consider the possibility to apply some approaches
and solutions of BGP in P2P environment.
NI N2 X N4 NI N2 X N4
IV. BYZANTINE GENERALS PROBLEM iN P2P ENVIRONMENT
Many types of P2P attacks exploit the interaction between P3 NI N2 Y N4 P4 NI N2 Y N4
the peers, coordinating and monitoring the activities of Ni N2 N3 N4 I J K L
independent nodes and ensuring robust, reliable
communication between nodes. NI N2 Z N4 NI N2 Z N4
The analogy between the behavior of malicious nodes in
P2P networks and actions of traitorous components in the case
of Byzantine failure is obvious. Consequences are the same: As we can see, in each matrix there are one row and
breaking down of consistent operation of the network nodes. one column containing the values different from other
So, according to many opinions, to resolve some security cells, and these values were sent by the third peer. So, it is
problems in P2P environment, it is possible to apply some possible to detect the traitor at once and to ignore this
approaches used in the case of Byzantine failure. But it's not node in the future.
so easy. We can meet some problems here. As it was noted
above, P2P networks present a very particular environment Classical solutions for BGP are efficient and precise,
with thousands and thousands of participants, where nodes and can be applied also in the case of incorrect lookup
join and leave the network permanently. So, a P2P network routing and other types of attack. However, the solutions
cannot be considered as a system where each component can presented by Lamport and Shostak are expensive in both
contact directly all others. Hence, existing solutions should be the amount of time and the number of required messages.
opportunely adapted for P2P networks. Let's analyze the In fact, their algorithms (OM(m) and SM(m)) [3] require
message paths of length up to t + 1 (where t is a number 1T01
of traitors). And consequently, the number of messages 16\
nodes send to each other is 140=
(n - )(n-2) ... (n-t1 =
n1!/t1) 120 N ; N=

i.e., roughly speaking, there is a factorial-like

dependence between the number of messages and the -

number of nodes. In P2P environment it will cause the 40A

increase of the traffic and the overload of the network, 20 =
because of a great quantity of nodes. 4 6 64 2% 1 4 4C96 1634 6T36 2044
The solution presented by Lewis and Saia reduces the 8 32 12B 512 2D48 8192 327 131072 24
number of messages [6]. This is achieved by using k
random sampling. But in this case we loose the accuracy
of Lamport's and Shostak's solutions, we cannot detect
who is traitor and the protocol is correct only with high
probability. And anyway, this algorithm involves all the
components of the network and the "final decision" is 3M-
based on an opinion of the majority. 280
But in reality one peer does not contact and does not 2
need to contact all the peers of the network, and it is not N.=i\
\=z
important for one given peer (user), that some peer (user) 160- = ==
in the network, having resources which he will be never E140 ==
interested in, is a traitor. In other words, usually, a fan of =
=1000 =A
heavy-metal does not search for classic music files. But it 80

is important for him to know who is a traitor among the 40

20
=
peers he contacts very often. For small groups and 16 1024 4096 16384 6536 262144
subgroups of peers (users) the classical solutions of BGP4
1 w , 1 o rf
1 ,11 * v 1 * * 1 ~ ~ ~~~
32 ~~~~~~~8 5242388
by Lamport and Shostak become efficient. The principles 128 512
kk
2048 8192 32768 131072

of creation of these peer groups can be different. The

network can be divided into subgroups in many senses,
such as: a group created by nodes which have the most
frequent contacts among them; a group of neighbor nodes
with "close" identifiers according to XOR-metric; groups
of peers individuated by common interests, etc.
One of the possible ways that we propose to
individuate the groups in the network can be based on the
concept of "Codat", originally described by JXTA
platform developers [8]. Codat is defined as a unit of
information shared and exchanged within a peer group.
Codat (Code/Data) means a content that could be either
code or data and is used as a placeholder for any types of
data. A Codat is uniquely identified via a unique CodatID
and can belong to only one peer group. Every Codat
contains a document (a content advertisement) that
represents the data it holds. For example, a group of
persons interested in cooking creates the peer group to
exchange the recipes and unites the resources of all the
peer members under the Codat with the keyword
"Cookery". And all the information about the available
resources has to be indicated in the content advertisement.
So, the Codat can be a key component to define interests
of different peer groups of the network in order to
individuate them in such manner.
So, if we can see a P2P network as a number of peer
groups individuated by some principle, we can reduce the
number of circulating messages.
Figure 2 - Diagrams of Nm change at k increased for t =1/3N
In this case each node launches the algorithm described
above only within its group. And then, the number of
messages circulating on the network during the process of
traitor detection using algorithms SM(m) by Lamport and
Shostak is:
Nm = ((ni - l)(ni - 2) ... (ni - ti - l)) xk (2)
where ni - the average number of nodes in one group
formed around one given node i; ti - the average number
of traitors in one group; k - the number of groups on the
network.
The diagrams show how the number Nm of messages
changes while the number k of groups increases, at
various network sizes (number of nodes in the network N
= 100; 1000; 10000; 100000; 1000000). Figure 1
illustrates this dependence in the case when the number of
traitors on the network is t = 1/5N and figure 2 in the case
when t = 1/3N (the worst case).
It should be noted that in this case all the groups are
completely separated, i.e. they do not have nodes in
common. In reality some groups may have some nodes in
common, i.e. the groups of neighbors (or of the most
frequent contacts) of a node may partially or completely
coincide with those of some others. In this case, a node
will repeatedly participate in the algorithm described
above, as many times as the number of different groups it
 belongs to. Then, reducing the number of messages on the [3] Leslie Lamport, Robert Shostak, Marshall Pease
network requires some reconsideration of messaging "The Byzantine Generals Problem", ACM
algorithms between nodes on the network, for example to Transactions on Programming Languages and
make the results of traitor's detection available to other Systems, Vol.4, No.3, July 1982, Pages 382 -
groups. 401
V. CONCLUSIONS AND FUTURE WORK [4] J.Garay, Y.Moses "Fully Polynomial Byzantine
Agreement for n > 3t Processors in t+1 Rounds,
In this paper we showed that it is possible to apply SIAM Journal of Computing, 27(1), 1998
successfully the classical solutions for BGP by Lamport
and Shostak in P2P environment. Also, it was illustrated [5] MORabin "Randomized Byzantine generals", In
that application of these solutions helps to reduce the 24th Annual Symposium on Foundations of
number of messages circulating on the network. But it Computer Science (FOCS'83), pages 403 - 409, Los
should be noted here, that we didn't take in consideration Alamitos, CA, USA, November 1982, IEEE
the case of partially coinciding groups yet. It should be Computer Society Press
the next step of our research. [6] Clifford Scott Lewis, Jared Saia "Scalable Byzantine
Also it is important to note, that in our example we Agreement",
deal with the malicious node who demonstrates poor htt*:Hwww.research.ibm.com/nipsO3workshop/abstra
behaviour regarding all the nodes. In the case of cts/lewis-saia.pdf
inconsistent behaviour it is not so easy to detect the . . .
traitor. When a malicious peer is more "cunning" and c7]pVtA. nekov k"OperAtio lsytem o ri bue
sends the real values to one part of the nodes, and to other CopTer newrks" (B.AHHpIXKBOLeHCHT HbIe
part of them sends arbitrary values, it becomes possible
to suspect one of the "good" nodes of malicious activity.
part~~~~~~~~
cHcTeM)
CHCTem available
a b
httP://,arallel.ru/krkov/
iin Russian
RsianT on on

At this point some mechanisms of reputation could be

successfully activated. [8] JXTA v2.0, Protocols Specification,
So, in our future work, we have to find answers for the httv:Hs-ecorsht//docbok/JXTAP ls.htm
following questions: api/doc/api/net/jxta/codat/Codat.htmI
* What other mechanisms can we use to individuate a
group?
* How can we optimize the algorithm of the traitor
detection process in the case of partially coinciding
groups?
* How can we integrate the solutions we have just presented
with some mechanisms of reputation evaluation?

REFERENCES
[1] Petar Maymounkov, David Mazieres "Kademlia: A
Peer-to-Peer Information System Based on the XOR
Metric", 1 St International Workshop on Peer-to-Peer
Systems, MIT, March 2002,
http://ademia.scs.csn
[2] Emil Sit, Robert Morris "Security Considerations for
Peer-to-Peer Distributed Hash Tables", 1 st
International Workshop on Peer-to-Peer Systems
(IPTPS'02), Cambridge, Massachusetts, March 2002