W 5 Lectures

Distributed Computing
Synchronization
Asad W. Malik
asad.malik{@}seecs.edu.pk
National University of Sciences and Technology (NUST)
Why synchronization?
❏ You want to catch a bus at 6:00 pm, but your watch is off by 15 minutes
❏ What if your watch is late by 15 minutes?
❏ You will miss the bus!
❏ What if your watch is Fast by 15 minutes?
❏ You will end up waiting for a longer time
❏ Time Synchronization is required in both cases

2
Synchronization In the Cloud
❏ Consider Airline reservation system
❏ Server A receives a client request to purchase last ticket on flight ABC 123
❏ Server A timestamps purchase using local clock 9h:15m:32.45s and logs it.
❏ Replies ok to client
❏ That was the last seat. Server A send message to Server B saying “flight full”
❏ B enters “Flight ABC 123 full” + its own local clock value (which reads
9h:10m:10.11s) into its log
❏ Server C queries A’s and B’s log. Is confused that a client purchased a ticket at A
after the flight became full at B
❏ This may lead to further incorrect actions by C 3
A BaseBall Example
Ten events, three entities - Pitcher, first base, third base and home
e1: pitcher throws ball to home
e2: ball arrives at home
e3: batter hits ball to pitcher
e4: batter runs to first base
e5: runner runs to home
e6: ball arrives at pitcher
e7: pitcher throws ball to first base
e8: runner arrives at home
e9: ball arrives at first base
e10: batter arrives at first base 4
A BaseBall Example
e2: ball arrives at home
e3: batter hits ball to pitcher
e4: batter runs to first base
e5: runner runs to home
e6: ball arrives at pitcher
e7: pitcher throws ball to first base
e8: runner arrives at home
A BaseBall Example
6
A BaseBall Example
e2: ball arrives at home ❏ Pitcher knows e1 happens before
e3: batter hits ball to pitcher e6, which happens before e7
e4: batter runs to first base ❏ Home plate umpire knows e2 is
e5: runner runs to home before e3, which is before e4,
e6: ball arrives at pitcher which is before e8,…….
e7: pitcher throws ball to first base ❏ Relationship between e8 and e9 is
e8: runner arrives at home unclear
Ways to Synchronize…...
❏ Send message from first base to home?

❏ Use a central timekeeper
❏ How long does this message take to arrive?

❏ Synchronize clocks before the game?
❏ Clock drift
8
Nothing like Perfect World…..!
❏ Messages always arrive, with propagation delay exactly d

❏ Sender sends time T in a message
❏ Receiver sets clock to T+d
❏ Real networks are unreliable
A T
B
T+d
9
Synchronization
Is it possible to synchronize all clocks in distributed system?
10
Clock Synchronization - Topics
❏ Physical clock synchronization

❏ Cristian's algorithm
❏ The Berkeley Algorithm
❏ What is logical clock?

❏ Why we need logical clock?
11
Clock Synchronization
❏ In single system, time is unambiguous

❏ Process A acquire time through system call
❏ Later process B acquire time through system call
A.time < B.time
❏ UNIX “make” program works
❏ large programs split into multiple files
❏ change in one file require one file to recompile
❏ increase speed
❏ In distributed systems, time is not trivial 12
Clock Synchronization
13
Physical Clock
❏ Computer timer is depend on quartz crystal

❏ Crystal oscillate at well defined frequency
❏ Frequency depend on kind of crystal, how crystal is cut, tension wire
Associate with crystal, two registers
❏ Counter
❏ holding register
❏ Each oscillation, decrement the counter, on zero interrupt generated
and counter reloaded from holding register
❏ Interrupt generated 60 times a second, interrupt is called clock tick 14
Physical Clock
❏ Cristian's Algorithm - Network Time Protocol

❏ The Berkeley Algorithm
15
Cristian’s Protocol
❏ Proposed in 1989, commonly used in many protocols

❏ Server is passive
❏ Server is equipped with accurate clock or receiver
❏ Client request the server for time
❏ The problem is message delay outdated the reported time
16
17
❏ A send a request to B with timestamped T1

❏ B in turn will record the time of receipt T2
❏ Return response with T3, piggyback with T2
❏ A records the time of the response arrival T4
❏ Assume propagation delay is roughly the same from A-B, B-A
18
19
Cristian’s Protocol - Network Time Protocol
❏ In NTP, B will also probe A for its current time

❏ The offset is computed as given above
❏ The delay estimation is calculated as
❏ delay = [(T2 - T1) + (T4 - T3) ]/2
❏ Eight pair of values are buffered (offset, delay)

❏ Finally taking the minimum value as the best estimation
20
The Berkeley Algorithm
❏ The Berkeley algorithm is a method of clock synchronization which

assumes no machine has an accurate time source
❏ It was developed by Gusella and Zatti at the University of California,
Berkeley in 1989
❏ intended for use within intranets
❏ The server process in the Berkeley algorithm, called the master,
periodically polls other slave processes.
21
❏ A master is chosen via an election process

❏ The master polls the slaves who reply with their time
❏ The master then averages the clock times
❏ Instead of sending the updated current time, the master then sends
out the amount (positive or negative) that each slave must adjust its
clock. This avoids further uncertainty due to RTT at the slave processes
22
23
24
25
Lesson learned…..
❏ Clocks never exactly synchronized - Clock drift

❏ Often inadequate for distributed systems
❏ might need totally-ordered events
27
Distributed Systems
Logical Clocks
Logical Clock
❏ Consider UNIX make program again

❏ How much source and object file is outdated OR
find object file is outdated
❏ Thats speak for the logical clock
29
Lamport’s Logical Clocks
❏ if two process do not interact, it is not necessary their clocks be

synchronized lack of synchronization would not be observable
❏ It does not matter all processes would agree on what time it is
rather they agree on the order in which event occurs
❏ In make program what matters is input.c is older or newer than
input.o Not their absolute time creation
30
❏ Define a relationship “happens-before”

❏ a ->b read a happens before b
❏ Happens before relation can be observed in two situations:
❏ 1). a and b are two events and a occurs before b, then a -> b is true
❏ 2). if a the event of message being sent by one process and b is the
event of message received by another process then a -> b is also true
31
❏ Happen-before is a transitive relationship

❏ Concurrent events
❏ Two events x and y happen in different processes do not exchange
messages
x -> y or y -> x
32
❏ Need a way to measure a notion of time, for every event ‘a’ we can
assign a time value C(a)
❏ if a -> b then C(a) < C(b) ’a’ and ‘b’ occur on same process
❏ ’a’ is sending of a message by one process and ‘b’ is receiving of a
message by another process then
❏ C(a) < C(b) must be assign, everyone agrees
❏ Lamport propose for assigning times to events
33
❏ At time 6, P1 sends message m1 to process p2

How long this will take?
❏ if message carries starting time, mean it took 10
ticks message m2 from p2 to p3 takes 16 ticks
❏ Consider message m3, it leave p3 at 60 and arrive
at p2 at 56?
❏ This is the situation that must be prevented

34
❏ Lamport’s algorithm follow happens-before

relation
❏ since m3 left at 60
❏ it must arrive at 61 or later
❏ on arrival if the receivers clock shows a value prior
to the time message was sent
❏ receiver fast forward its clock
35
Lamport’s Logical Clocks Implementation
❏ Lamport’s logical clocks implementation

❏ Each process maintain a local counter Ci
1). Before executing any event, Pi executes Ci = Ci + 1

2). When Pi send a message m to Pj its set m’s timestamp equal to Ci
after executing previous step
3).Upon receipt of a message m, Pj adjusts its own local counter as Cj
max {Cj , m(ts) } and execute the step-1 and deliver the message to the
application
36
37
How to solve this issue?
38
How to solve this issue?
39
Lamport’s Logical Clocks - Class Activity
40
41
42
Case Study - Synchronization
In some situations, an additional requirement is desirable, i.e. no two events

ever occur at exactly at the same time. How will your implementation
ensure such a behavior with n process; where 0 < n < 1000
43
Assume that each process’s logical clock is set to 0
44
❏ A timestamp of 1 is associated with events a,
e, j in processes P1, P2, P3 respectively
❏ A timestamp of 2 is associated with events b, k
in processes P1, P3 respectively
❏ The times may be the same but the events are
distinct
❏ We would like to create a total order of all
events i.e. for an event a, b we would like to
say that either a -> b or b -> a
45
Example: Totally-Ordered Multicast
❏ Scenario
❏ Replicated accounts in New York(NY) and San
Francisco(SF)
❏ Two transactions occur at the same time and
multicast
❏ Current balance: $1,000
❏ Add $100 at SF
❏ Add interest of 1% at NY
❏ If not done in the same order at each site then
one site will record a total amount of $1,111 and
the other records $1,110.
46
Example: Totally-Ordered Multicast
❏ We must ensure that the two update operations are performed in the same
order at each copy
❏ Although it makes a difference whether the deposit is processed before the
interest update or the other way around, it does matter which order is followed
from the point of view of consistency
❏ We need totally-ordered multicast, that is a multicast operation by which all
messages are delivered in the same order to each receiver.
❏ NOTE: Multicast refers to the sender sending a message to a collection of
receivers.
47
Algorithm: Totally-Ordered Multicast
1) Update message is time stamped with sender’s logical time

2) Update message is multicast (including sender itself)
3) When message is received
4) put into local queue
5) Ordered according to timestamp,
6) Multicast acknowledgement
48
❏ Message is delivered to applications only when

❏ It is at head of queue
❏ It has been acknowledged by all involved processes
❏ Pi sends an acknowledgement to Pj if
❏ Pi has not made an update request
❏ Pi’s identifier is greater than Pj’s identifier
❏ Pi’s update has been processed;
❏ Lamport algorithm (extended for total order) ensures total
ordering of events 49
50
❏ If P2 gets (n,1.2) before (m,1.1) does it

still multicast an acknowledgement for
(n,1.2)?
❏ At this point, how does P2 know that
there are other updates that should be
done ahead of the one it issued?
❏ Does P2 multicast an
acknowledgement for (m,1.1) when it
receives it? 51
52
❏ To summarize, the following messages have been sent:

❏ P1 and P2 have issued update operations
❏ P1 has multicasted an acknowledgement message for (m,1.1)
❏ P2 has multicasted acknowledgement messages for (m,1.1), (n,1.2)
❏ P1 and P2 have received an acknowledgement message from all
processes for (m,1.1)
❏ Hence, the update represented by m can proceed in both P1 and
P2
53
❏ When P1 has finished with m, it
can then proceed to multicast an
acknowledgement for (n,1.2).
❏ When P1 and P2 both have
received this acknowledgement,
then it is the case that
acknowledgements from all
processes have been received
for (n,1.2).
❏ At this point, it is known that the
update represented by n can
proceed in both P1 and P2. 54
55
What if there was a third process e.g., P3 that issued an update (call it o) at
about the same time as P1 and P2.
The algorithm works as before.
P1 will not multicast an acknowledgement for o until m has been done.
P2 will not multicast an acknowledgement for o until n has been done.
Since an operation can’t proceed until acknowledgements for all processes
have been received, o will not proceed until n and m have finished
56
Totally-Ordered Multicast
57
Vector Clocks
58
Vector Clocks
❏ Problem with Lamport’s algorithms is based on happens before relation

❏ In Lamport clock nothing can be said about relationship between two
events a and b by looking C(a) and C(b)
59
Motivation to Vector Clock
Logical clock - Event s happens before event t => the logical clock value of s
is smaller than the logical clock value of t.
Vector clock - Event s happens before event t means the vector clock value
of s is “smaller” than the vector clock value of t
Each event has a vector of n integers as its vector clock value
❏ v1 = v2 if all n fields same

❏ v1 ≤ v2 if every field in v1 is less than or equal to the corresponding
field in v2
60
Vector Clocks
Lamport clock do not capture causality
61
Vector Clock
Lamport clock do not capture causality
This causality can be captured through

vector clock
62
Vector Clock Concept
❏ A vector clock VC(a) assigned to an event a has the property that if

VC(a) < VC(b) for some event b
❏ Then event is known to casually precede event b
63
Vector Clock Construction
Vector clock is constructed by letting each process Pi maintain a vector VCi with two
properties
1) VCi[i] is number of events that have occurred so far at Pi. VCi[i] is logical clock at
Pi
2) VCi[j] = k then Pi knows that k events have occurred at Pj. Pi has a knowledge of
local time at Pj
First property is maintained by incrementing VCi[i] at the occurrence of each event
The second is maintained by piggybacking vectors along with messages that are sent 64
Vector Clock Algorithm
1) Before executing an event Pi executes VCi[i] ← VCi[i]+1

2) When process Pi sends a message m to Pj, it sets m’s vector timestamp
ts(m) equal to VCi after having executed the previous step
3) Upon receipt of a message m, process Pj adjusts its own vector by
setting VCj[k] ← max{VCj[k], ts(m)[k]} for each k, after which it
executes the first step and delivers the message to the application
65
Vector Clock
66
Vector Clock
A (1,0,0)
67
Vector Clock
A (1,0,0)
(2,0,0)
B (2,0,0)
68
Vector Clock
A (1,0,0)
(2,0,0)
B (2,0,0)
C(2,1,0)
(2,1,0)
69
Vector Clock
A (1,0,0)
(2,0,0)
B (2,0,0)
(2,2,0)
C(2,1,0)
(2,1,0)
D(2,2,0)
70
Vector Clock
A (1,0,0)
(2,0,0)
B (2,0,0)
(2,2,0)
C(2,1,0)
(2,1,0)
D(2,2,0) (0,0,1)
E(0,0,1) (2,2,2)
F(2,2,2) 71
Find the
concurrent
Vector Clock
events
A (1,0,0)
(2,0,0)
B (2,0,0)
(2,2,0)
C(2,1,0)
(2,1,0)
D(2,2,0) (0,0,1)
E(0,0,1) (2,2,2)
F(2,2,2) 72
Vector Clock - Obeying Causality
A (1,0,0) -> B (2,0,0)

(2,0,0)
(1,0,0) < B (2,0,0)
(2,2,0)
E(0,0,1) -> F(2,2,2)
(2,1,0)
(0,0,1) < (2,2,2) (0,0,1)
(2,2,2)
A(1,0,0) ? E(0,0,1)
73
Class Activity
A B C D E
E’
F G
H I J
74
Class Activity
75
Causality
If a->b then event a can affect event b
Concurrency
If neither a->b nor b->a then one event cannot affect the other
Partial Ordering
Causal events are sequenced
Total Ordering
All events are sequenced
76
77
Algorithms for Causal ordering of Messages
1) BSS- Birman-Schiper Stephenson Algorithm

2) SEC- Schiper-Eggli-Sandoz Algorithm
3) Matrix Algorithm
78
BSS- Birman-Schiper Stephenson Algorithm
❏ Broadcast based: a message sent is received by all other processes

❏ Deliver a message to a process only if the message preceding it has
been delivered to the process
❏ Otherwise, buffer the message
❏ No clock increment on receiving
❏ Accomplished by using a vector accompanying the message
79
1) Process Pi increments the vector time VTpi[i], time stamps and broadcasts the message m.
VTpi[i] - 1 denotes the number of messages preceding m.
2) Pj != Pi receives m. m is delivered when:
a. VTpj[i] == VTm[i] - 1
b. VTpj[k] >= VTm[k] for all k in {1,2,..n} - {i}, n is the total number of processes. Delayed
message are queued in a sorted manner.
c. Concurrent messages are ordered by time of receipt.
3) When m is delivered at Pj, VTpj updated according Rule 2 of vector clocks
3(a) : Pj has received all Pi’s messages preceding m
3(b): Pj has received all other messages received by Pi before sending m
80
81
82
Class Activity- BSS
83

W 5 Lectures

Uploaded by

Document Information

Original Description:

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

W 5 Lectures

Uploaded by

Copyright:

Available Formats

Distributed Computing

❏ You will miss the bus!

❏ What if your watch is Fast by 15 minutes?

❏ You will end up waiting for a longer time

❏ Time Synchronization is required in both cases

❏ Send message from first base to home?

❏ How long does this message take to arrive?

❏ Messages always arrive, with propagation delay exactly d

Is it possible to synchronize all clocks in distributed system?

❏ Physical clock synchronization

❏ The Berkeley Algorithm

❏ What is logical clock?

❏ In single system, time is unambiguous

❏ Computer timer is depend on quartz crystal

❏ Cristian's Algorithm - Network Time Protocol

❏ Proposed in 1989, commonly used in many protocols

❏ A send a request to B with timestamped T1

❏ In NTP, B will also probe A for its current time

❏ Eight pair of values are buffered (offset, delay)

❏ The Berkeley algorithm is a method of clock synchronization which

❏ A master is chosen via an election process

❏ Clocks never exactly synchronized - Clock drift

❏ Consider UNIX make program again

❏ if two process do not interact, it is not necessary their clocks be

❏ Define a relationship “happens-before”

❏ Happen-before is a transitive relationship

❏ Lamport propose for assigning times to events

❏ At time 6, P1 sends message m1 to process p2

❏ This is the situation that must be prevented

❏ Lamport’s algorithm follow happens-before

❏ Lamport’s logical clocks implementation

1). Before executing any event, Pi executes Ci = Ci + 1

In some situations, an additional requirement is desirable, i.e. no two events

1) Update message is time stamped with sender’s logical time

❏ Message is delivered to applications only when

❏ If P2 gets (n,1.2) before (m,1.1) does it

❏ To summarize, the following messages have been sent:

❏ Problem with Lamport’s algorithms is based on happens before relation

❏ v1 = v2 if all n fields same

Lamport clock do not capture causality

Lamport clock do not capture causality

This causality can be captured through

❏ A vector clock VC(a) assigned to an event a has the property that if

First property is maintained by incrementing VCi[i] at the occurrence of each event

1) Before executing an event Pi executes VCi[i] ← VCi[i]+1

A (1,0,0) -> B (2,0,0)

1) BSS- Birman-Schiper Stephenson Algorithm

❏ Broadcast based: a message sent is received by all other processes

You might also like