Distributed Hash Tables

David Tam Patrick Pang

Presentation Outline
What is DHT (Distributed Hash Table)?

Why DHTs?
Applications How lookup works? Alternatives to DHTs Performance Routing Performance Load Balancing Security Routing Attack Security Inconsistent Behaviour Comparison to Other Facilities Current Research Projects Conclusion

What is DHT?
Distributed application put(key, data) get (key) Distributed hash table node . data

node

node

DHT provides the information look up service for P2P applications.

Nodes uniformly distributed across key space Nodes form an overlay network Nodes maintain list of neighbours in routing table Decoupled from physical network topology
(Figure adopted from Frans Kaashoek)

Why DHTs?
Why Middleware? Simplifies the development for large-scale distributed Apps Better security and robustness Simple API

Why Do We Need DHTs? Simplifies the development for large-scale distributed Apps Better security and robustness Simple API Exploits P2P resources

Applications
Anything that requires a hash table Databases, FSes, storage, archival Web serving, caching Content distribution Query & indexing Naming systems Communication primitives Chat services Application-layer multi-casting Event notification services Publish/subscribe systems ?

How lookup works?

Example: Chord [Stoica et. al.] Finger Table for Node 2 15 14 13 12 4 5 10 7 9 8 6 0

1
2 3

start interval 3 [3,4)

succ. 5

4
6 10

[4,6)
[6,10) [10,2)

5
7 10
11

How lookup works?

Example: Chord Finger Table for Node 10 15 14 13 12 4 5 10 7 9 8 6 0

1
2 3

start interval 11 [11,12)

succ. 12

12
14 2

[12,14)
[14,2) [2,10)

12
14 2
11

How lookup works?

Example: Chord Finger Table for Node 10 15 14 13 12 4 5 10 7 9 8 6 0

1
2 3

start interval 11 [11,12)

succ. 12

12
14 2

[12,14)
[14,2) [2,10)

12
14 2
11

How lookup works?

Example: Chord Finger Table for Node 14 15 14 13 12 4 5 10 7 9 8 6 0

1
2 3

start interval 15 [15,0)

succ. 15

0
2 6

[0,2)
[2,6) [6,13)

1
2 7
11

How lookup works?

Example: Chord Finger Table for Node 14 15 14 13 12 4 5 10 7 9 8 6 0

1
2 3

start interval 15 [15,0)

succ. 15

0
2 6

[0,2)
[2,6) [6,13)

1
2 7
11

How lookup works?

Example: Chord 15 14 Now Node 2 can retrive information for key 0 from Node 1. 0

1
2 3 4 5

12

11
10 7 9 8 6

Alternatives to DHTs
Distributed file system Centralized lookup P2P flooding queries
Server

N1 N4
Target

N2
N6

N3

Start

N10

N9
Client

N7 N8

Client

Internet

N1

N4
Client

N2 DB N6

Target

N3
N8

Start

N10

Client Server

N9

N7

(Figures adopted from Frans Kaashoek)

Performance -- Lookup
Purpose -- to locate a target node Each step, try to get closer to locating target node Ask a closer neighbour Performance & scalability tied directly to lookup algorithm

2 Aspects to Performance Path latency Lookup path length (# hops)

2 Aspects to Scalability size of routing table O(log N) lookup path length O(log N)

3 Techniques proximity lookup proximity neighbour selection geographic layout

Performance -- Load Balancing

Issues Hot-spots Content Lookup Heterogeneous nodes & paths System flux
Solution Replication is the key Also good for fault-tolerance Cache lookup answers backwards along path

Security Incorrect Lookup (1)

When asked for the next hop, give a wrong answer
Finger Table for Node 2 start 3 4 6 10 interval [3,4) [4,6) [6,10) [10,2) succ. 5 5 7 10

0 15 14

1 2 3 4 5

13
12 11 10 7 9 8

Node 2 to Node 10: Please tell me how to reach key 0 .

Security Incorrect Lookup (2)

When asked for the next hop, give a wrong answer
Finger Table for Node 10 start 11 12 14 2 interval [11,12) [12,14) [14,2) [2,10) succ. 12 12 14 2

0 15 14

1 2 3 4 5

13
12 11 10 7 9 8

Node 2 to Node 10: Please tell me how to reach key 0 .

Node 10 answers: ask Node 14

Security Incorrect Lookup (3)

When asked for the next hop, give a wrong answer
Finger Table for Node 14 start 15 0 2 6 interval [15,0) [0,2) [2,6) [6,13) succ. 15 1 2 7

0 15 14

1 2 3 4 5

13
12 11 10 7 9 8

Node 2 to Node 14: Please tell me how to reach key 0 .

Node 14 answers: ask Node 10

Security Incorrect Lookup (4)

Solution [Sit and Morris]: Define verifiable system invariant Allow the querier to observe lookup progress Our idea how this can be implemented: Concretely, using an integral monotonically decreasing quantity to implement the idea of progress. The concept of monotonically decreasing quantity has been used in program construction guaranteeing total correctness. [Parnas]

Security Inconsistent Behaviour

Inconsistent Behaviour, i.e., lie intelligibly Sybil attack [Kaashoek] Solution 1: public key solution

Security Inconsistent Behaviour

Inconsistent Behaviour, i.e., lie intelligibly Sybil attack [Kaashoek] Solution 1: public key solution

Solution 2: Byzantine Protocol

Byzantine Generals Problem: How to find out the traitors among the Generals? [Lamport]

Security Inconsistent Behaviour

Inconsistent Behaviour, i.e., lie intelligibly Sybil attack [Kaashoek] Solution 1: public key solution

Solution 2: Byzantine Protocol

Commander Byzantine Generals Problem: How to find out the traitors among the Generals? [Lamport] Lieutenant 1 he said retreat attack attack

Lieutenant 2

Security Inconsistent Behaviour

Inconsistent Behaviour, i.e., lie intelligibly Sybil attack [Kaashoek] Solution 1: public key solution

Solution 2: Byzantine Protocol

Commander Byzantine Generals Problem: How to find out the traitors among the Generals? [Lamport] Lieutenant 1 he said retreat attack retreat

Lieutenant 2

Comparison to Other Facilities

Facility
DHT Centralized Lookup P2P flooding queries Distributed FS Facility DHT Centralized Lookup Distributed FS

Abstraction Easy Use/Prg Scalability

high medium medium low high medium high medium high low low medium

Load-Balance
yes no no no

Fault-Tolerance Self-Org Admin high low medium yes no yes no low medium low high

P2P flooding queries depends

Research Projects
Iris security & fault-tolerance US Govt Chord circular key space Pastry circular key space Tapestry hypercube space CAN n-dimensional key space Kelips n-dimensional key space DDS -- middleware platform for internet service construction -- cluster-based -- incremental scalability

Summary
Good middleware platform
Exploits P2P networks An exciting new research area

References
Lamport, Leslie et. al. The Byzantine Generals Problem
Sit, Emil, Morris, Robert. Security Considerations for Peerto-Peer Distributed Hash Tables Kaashoek, Frans. Distributed Hash Tables Building largesacle, robust distributed applications Stoica, Ion et. al. Chord: A scalable peer-to-peer lookup service for Internet applications Parnas, D. L. Connecting Theory to Practice: Software Engineering Programme