Distributed Computing

Lecture 2 : Decentralized Organization

Slides adopted from the instructional material of the textbook

Distributed Systems: Principles and Paradigms by Andrew S. Tanenbaum, Maarten van Steen.
• Distributed system concepts
• Distributed system design goals
• DG 1: Resource Sharing
• DG 2 : Distributed Transparency
• DG 3 :Openness
• DG 4:Scalability
• Architectures Styles
• Today
• Decentralized Organization: P2P
• Chord
• DHT
Decentralized Organizations : P2P
 Decentralized Architectures
There is a tremendous growth in usage of peer-to-peer
systems.
 Structured P2P:
nodes are organized following a specific distributed
data structure (Ex. Ring)
 Unstructured P2P:
nodes have randomly selected neighbors
 Hybrid P2P:
some nodes are appointed special functions in a well-
organized fashion

In all cases, we are dealing with overlay networks: data is

routed over connections setup between the peers (e.g.
application-level multicasting)
 Structured P2P Architectures
Basic idea: Organize the nodes in a structured overlay
network such as a logical ring, and make specific nodes
responsible for services based only on their ID.

 The system provides an

operation lookup(key) that
efficiently routes
the lookup request to the
associated node.

 The mapping of data items

onto nodes in Chord.
 Structured Peer-to-Peer Architectures
Content Addressable Network:
Organize nodes in a d-dimensional space and let every node take
the responsibility for data in a specific region. When a node joins,
split a region.

The mapping of data items onto nodes in CAN. Splitting a region when a node joins
Unstructured Peer-to-Peer Architectures
Many unstructured P2P systems attempt to maintain a random
graph.

Basic principle:
Each node is required to contact a randomly selected other
node:
•Let each peer maintain a partial view of the network,
consisting of c other nodes
• Each node P periodically selects a node Q from its partial view
•P and Q exchange information and exchange members
from their respective partial views

It turns out that, depending on the exchange, randomness, but

also
robustness of the network can be maintained.
Hybrid P2P: Superpeers
Selecting some nodes to do specific work: superpeer.

Examples:
-Peers maintaining an index
(for search)
-Peers monitoring the state
of the network
-Peers being able to setup
connections
A hierarchical organization of nodes into a
superpeer network.
DHT: Distributed Hash Tables
• Principles
• Chord DHT
• Logical ring
• Finger table
• Lookup
• Examples
• (Reading: Section 5.2.3 of the textbook)
Chord an Example of DHT

10/71
1
How to Construct a DHT (Chord)?

• Use a logical name space, called the identifier space, consisting of identifiers
{0,1,2,…, N-1}
15 0 1
• Identifier space is a logical ring modulo N.
14
2

13 3

12 4

11 5

10 6

9 7
8
1
How to Construct a DHT (Chord)?

• Use a logical name space, called the identifier space, consisting of identifiers
{0,1,2,…, N-1}
15 0 1
• Identifier space is a logical ring modulo N.
14
• Every node picks a random identifier 2
though Hash H.
13 3
• Example:
 Space N=16 {0,…,15} 12
 Five nodes a, b, c, d, e 4
 H(a) = 6
 H(b) = 5 11 5
 H(c) = 0
 H(d) = 11
 H(e) = 2 10 6

9 7
8
2
Successor ...

• The successor of an identifier

is the first node met going in
clockwise direction starting at 15 0 1
the identifier.
14
2

13 3

12 4

11 5

10 6

9 7
8
2
Successor ...

• The successor of an identifier

is the first node met going in
clockwise direction starting at 15 0 1
the identifier.
14
2
•succ(x): is the first node on the 13 3
ring with id greater than or
equal x.
 Succ(12) = 0 12 4
 Succ(1) = 2
 Succ(6) = 6
11 5

10 6

9 7
8
2
Connect the Nodes

• Each node points to its

successor.
 The successor of a node n is 15 0 1
succ(n+1).
 0’s successor is succ(1) = 2
 2’s successor is succ(3) = 5 14
2
 5’s successor is succ(6) = 6
 6’s successor is succ(7) = 11 13
 11’s successor is succ(12) = 0 3

12 4

11 5

10 6

9 7
8
3
Where to Store Data?

•Use globally known hash function, H.

• Each item <key,value> gets identifier
H(key) = k.
 H(''Almaz'') = 12
 H(''Challa'') = 2
 H(''Hagos'') = 9
 H(''Obang'') = 14
 H(''Tizita'') = 4
3
Where to Store Data?

•Use globally known hash function, H.

• Each item <key,value> gets identifier 15 0 1
H(key) = k.
14
 H(''Almaz'') = 12 Obang
Challa 2
 H(''Challa'') = 2
 H(''Hagos'') = 9 13 3
 H(''Obang'') = 14
 H(''Tizita'') = 4 Almaz
12 Tizita

11 5

Hag
10 os
6

9 7
8
3
Where to Store Data?

• Use globally known hash function, H.

• Each item <key,value> gets identifier 15 0 1

H(key) = k.
14
2
 H(''Almaz'') = 12
Challa
Obang

 H(''Challa'') = 2
 H(''Hagos'') = 9 13 3
 H(''Obang'') = 14
 H(''Tizita'') = 4 12
Almaz Tizita

• Store each item at its successor. 11 5

Hag
10 os
6

9 7
8
3
Where to Store Data?

• Use globally known hash function, H.

• Each item <key,value> gets identifier 15 0 1

H(key) = k. Obang

14 Almaz
2
 H(''Almaz'') = 12 Challa

 H(''Challa'') = 2
 H(''Hagos'') = 9 13 3
 H(''Obang'') = 14
 H(''Tizita'') = 4 12 4
Tizita

• Store each item at its successor. 11

Hag
os 5

10 6

9 7
8
Lookup?

20/71
 Lookup?

• To lookup a key k
Calculate H(k)
Follow succ pointers until item k is get(Hag
found 15 0 1 os)
Obang

14 Almaz
2
Challa

13 3

12 4
Tizita

Hag
11 os 5

10 6

9 7
8
 Lookup?

• To lookup a key k
Calculate H(k)
Follow succ pointers until item k is get(Hag
found 15 0 1 os)
• Example Obang

Lookup ''Hagos'' at node 2 14 Almaz

2
 H(''Hagos'')=9 Challa

Traverse nodes: 13
• 2, 5, 6, 11 (BINGO) 3
Return ''Jinka'' to initiator
12 4
Tizita

Key Value Hag

11 os 5
Hagos Jinka
10 6

9 7
8
 Lookup?
// ask node n to find the successor of id
procedure n.findSuccessor(id) {
if (predecessor  nil and id  (predecessor, n]) then return n
else if (id (n, successor]) then
return successor
else // forward the query around the circle
return successor.findSuccessor(id)
}

• (a, b] the segment of the ring moving clockwise from but not including a until and including b.
• n.foo(.) denotes an RPC of foo(.) to node n.
• n.bar denotes and RPC to fetch the value of the variable bar in node n.
 Put and Get

procedure n.put(id, value) {

s = findSuccessor(id)
s.store(id, value)
}

procedure n.get(id) {
s = findSuccessor(id)
return s.retrieve(id)
}

• PUT and GET are nothing but lookups!!

24/73
How can we improve this?

25/71
 Cost of Lookup Operations

• If only the pointer to succ(n+1) is used

 Worst case lookup time is O(N), for N nodes
15 0 1
Obang

14 Almaz
2
Challa

13 3

12 4
Tizita

Hag
11 os 5

10 6

9 7
8
 Speeding up Lookups

• Finger/routing table:
Point to succ(n+1)
Point to succ(n+2)
Point to succ(n+4) 15 0 1
Point to succ(n+8) 14
… 2
Point to succ(n+2 )
M1

13 3
• Distance always halved to the
destination. 12 4

11 5

10 6

9 7
8
 Speeding up Lookups

• Size of routing tables is logarithmic.:

 Routing table size: M, where N = 2^M.
15 0 1
• Every node n knows
successor(n + 2^(i1)) 14
2
for i = 1... M
13 3
• Routing entries = log2(N)
 log2(N) hops from any node to 12 4
any other node

• Example: Log2(1000000)≈20 11 5

10 6

9 7
8
 DHT Lookup
// ask node n to find the successor of id
procedure n.findSuccessor(id) {
if (predecessor  nil and id  (predecessor, n]) then return n
else if (id (n, successor]) then
return successor
else // forward the query around the circle
return successor.findSuccessor(id)
}
 DHT Lookup
// ask node n to find the successor of id
procedure n.findSuccessor(id) {
if (predecessor  nil and id  (predecessor, n]) then return n
else if (id (n, successor]) then
return successor
else // forward the query around the circle
return successor.findSuccessor(id)
}

closestPrecedingNode(id)
 DHT Lookup
// ask node n to find the successor of id
procedure n.findSuccessor(id) {
if (predecessor  nil and id  (predecessor, n]) then return n
else if (id (n, successor]) then
return successor
else { // forward the query around the circle
m := closestPrecedingNode(id)
return m.findSuccessor(id)
}
}
// search locally for the highest predecessor of id
procedure closestPrecedingNode(id) {
for i = m downto 1 do {
if (finger[i] (n, id)) then
return finger[i]
}
return n
}
 Distributed Hash Tables
Example

Resolving key 26 from node 1, and key 12 from node 28 in a Chord system.
Example: Chord DHT
Chord finger tables
 Chord lookup examples
o IF key > all FT entries:
a) If no wrapping in Finger table, send to FT[-1]

o IF key < all FT entries:

a)key > p, send to FT[0]
(as close as possible)
Lookup(3) @ 1: 3 < all entries in 1, but
3 > 1, send to first entry (4)

b)key < p, send to FT[-1]

(as far as possible)
E.g., Lookup(3) @ 9:
3 < all entries in 9 but 3 < 9:
send to last entry (28)

(see next slide for this example)

oIF wrapping happens at FTp[i]:

if key > p:
- send to FTp[i-1] if any, (should i?)
- otherwise, send to FTp[i]

else (key < p):

let the other rules decide

lookup(13) @ 21: send to last entry

 Node Joins – with Finger Tables

finger table keys

start int. succ. 6
1 [1,2) 1
2 [2,4) 3
4 [4,0) 06

finger table keys

0 start int. succ. 1

7 1 2 [2,3) 3
3 [3,5) 3
5 [5,1) 06
finger table keys
start int. succ. 6 2
7 [7,0) 0
0 [0,2) 0 finger table keys
2 [2,6) 3
5 3 start int. succ. 2
4 [4,5) 06
4 5 [5,7) 06
7 [7,3) 0
 Node Departures – with Finger Tables

finger table keys

start int. succ.
1 [1,2) 13
2 [2,4) 3
4 [4,0) 06

finger table keys

0 start int. succ. 1

7 1 2 [2,3) 3
3 [3,5) 3
5 [5,1) 06
finger table keys
start int. succ. 6 6 2
7 [7,0) 0
0 [0,2) 0 finger table keys
2 [2,6) 3
5 3 start int. succ. 2
4 [4,5) 6
4 5 [5,7) 6
7 [7,3) 00

37
Chord lookup example