Chapter4 ConcurrentDataStructures

Synchronization Algorithms
and Concurrent Programming

Gadi Taubenfeld
Chapter 4
Blocking and Non-blocking Synchronization
Version: June 2014

Chapter 4 Synchronization Algorithms and Concurrent Programming 1
Gadi Taubenfeld © 2014
Synchronization Algorithms
and Concurrent Programming
ISBN: 0131972596, 1st edition
A note on the use of these power-point slides:

I am making these slides freely available to all (faculty, students, readers).
They are in PowerPoint form so you can add, modify, and delete slides and slide
content to suit your needs. They obviously represent a lot of work on my part.
In return for use, I only ask the following:
q That you mention their source, after all, I would like people to use my book!
q That you note that they are adapted from (or perhaps identical to)
my slides, and note my copyright of this material.
Thanks and enjoy!

Gadi Taubenfeld
All material copyright 2014
Gadi Taubenfeld, All Rights Reserved
To get the most updated version of these slides go to:

http://www.faculty.idc.ac.il/gadi/book.htm
Chapter 4
Blocking and Non-blocking
Synchronization
4.1 Synchronization Primitives

4.2 Collision Avoidance using Test-and-set Bits
4.3 The Ticket Algorithm using RMW Objects
4.4 Local Spinning using Strong Primitives
4.5 Concurrent Data Structures
These topics are
-- Progress Conditions
covered in this
-- Consistency Conditions presentation
-- Non-blocking Queue
-- Memory Barriers
4.6 Semaphores
4.7 Monitors
4.8 Fairness of Shared Objects

Concurrent Data Structures
Using locks Without locks
P1 P2 P3 P1 P2 P3
data structure data structure

data structure

Concurrent Data Structures
Using locks Without locks
P1 P2 P3 P1 P2 P3

data structure
• simple programming model • resilient to failures, etc.

• false conflicts • often complex
• fault-free solutions only • memory consuming
• sequential bottleneck • sometime -- weak progress cond.
Progress Conditions
Section 4.5.1
Synchronization Algorithms and Concurrent Programming 6

Chapter 4
Progress Conditions
Using locks lock-free
P1 P2 P3 P1 P2 P3

data structure
e s
li ur
o fa
n
deadlock starvation FIFO obstruction non-blocking wait
(livelock) freedom (+ df ) freedom freedom
freedom
Obstruction-freedom
P1 P2 P3 P4
Done

Non-blocking
P1 P2 P3 P4
Done

Wait-freedom
P1 P2 P3 P4

Wait-freedom
P1 P2 P3 P4
Done Done Done Done

Lock-free Data Structures
Obstruction-freedom Non-blocking Wait-freedom
• too weak progress condition • strong enough • strong/desirable

• not complex • not so complex • complex/less efficient

Consistency Conditions
How do we define the correctness of a concurrent object ?
Ø Linearizability
Ø Sequential Consistency
Ø …
Synchronization Algorithms and Concurrent Programming

Chapter 4 13
Why are consistency conditions important
Ø Formal specification of concurrent objects
Ø Help to understand possible behaviors of an implementation

of a concurrent object and to reason about correctness
Ø Clarifies what optimizations are acceptable

Chapter 4 14
Example: Concurrent Queue
q.enq( )
q.deq( )

Chapter 4 15
Sequential Execution
Is this execution reasonable? Yes
P1 q.enq q.deq
P2 q.enq q.deq
time

Chapter 4 16
Concurrent Execution
Is this execution reasonable? ???
P1 q.enq q.deq
P2 q.enq q.deq
time

Chapter 4 17
P1 q.enq q.deq
P2 q.enq q.deq
time

Chapter 4 18
P1 q.enq q.deq
P2 q.enq q.deq
time

Chapter 4 19
Sequential Specification of an object
Queue: In sequential executions, the enqueue operation
inserts a value to the queue and the dequeue operation
returns and deletes the oldest value in the queue.
Sequential
specification

Chapter 4 21
Linearizability
for an object
Each concurrent execution is Each sequential execution

“equivalent’’ to an execution in is included in the
the sequential specification sequential specification
concurrent Sequential
executions executions
All executions
Chapter 4 22
Linearizability
for an object
Ø Need to preserve real time order
Ø Each operation should appear to “take effect” instantaneously

at some moment between its invocation and response
Ø Object is correct if this “sequential” behavior is correct
P1 q.enq q.deq
P2 q.enq q.deq

Chapter 4 23
Linearizability
Ø Permits programmers to specify and reason about concurrent

objects using known techniques from the sequential domain
Ø provides the illusion that each operation applied by concurrent

processes takes effect instantaneously between its invocation
and its response

Chapter 4 24
Terminology
Invocation Linearization Response

Point
q.enq

Chapter 4 25
Linearizability
Is this execution linearizable? Yes
P1 q.enq q.deq
P2 q.enq q.deq
time

Chapter 4 26
Linearizability
Is this execution linearizable? Yes X 2
P1 q.enq q.deq
P2 q.enq q.deq
time

Chapter 4 27
Linearizability
Is this execution linearizable? No, BUT …
P1 q.enq q.deq
P2 q.enq q.deq
time

Chapter 4 28
Linearizability
P1 q.enq q.deq
P2 q.enq q.deq
time

Chapter 4 29
Linearizability
Sequential Consistency
P1 q.enq q.deq
P2 q.enq q.deq
time

Chapter 4 30
Ø No need to preserve real time order

– Cannot re-order operations done by the same thread
– Can re-order non-overlapping operations done by different
threads
Ø Each operation should “take effect” instantaneously

between invocation and response events
Ø Object is correct if this “sequential” behavior is

correct

Chapter 4 31
Is this execution sequential consistent? Yes
P1 q.enq q.deq
P2 q.enq
time

Chapter 4 32
Is this execution sequential consistent? Yes
Its also linearizable!
P1 p.enq
P2 p.enq p.deq
time

Chapter 4 33
Is this execution sequential consistent?
P1 p.enq
P2 p.enq p.deq
time

Chapter 4 34
Is this execution sequential consistent? No !!!
P1 q.enq p.enq q.deq
P2 p.enq q.enq p.deq
time

Chapter 4 35
Is this execution sequential consistent? No !!!
P1 q.enq p.enq q.deq
P2 p.enq q.enq p.deq
time

Chapter 4 36
Compositionality
Theorem: Sequential consistency is not compositional.
Proof: The example from the previous slide.
Theorem: Linearizability is compositional.
Proof: ...

Chapter 4 37
A Non-blocking Concurrent Queue Algorithm
Section 4.5.2

Chapter 4
Implementing a Non-blocking Queue
enqueue dequeue

Implementing a Non-blocking Queue
Empty queue
tail head
Dummy Node

Enqueue
lnode tail head

2 concurrent enqueue operations
What can go wrong when using registers?
tail head

Dequeue
tail head

2 concurrent dequeue operations
What can go wrong when using registers?
tail head

Enqueue + Dequeue
(Non-empty queue)
tail head

Enqueue + Dequeue
(Empty queue)
tail head

Enqueue + Dequeue
(Empty queue)
tail head
Lost tail !!!
The dequeuer must advance tail before redirecting head

Impossibility Result
Theorem: There is a wait-free consensus algorithm for 2

processes, using queues and atomic registers.
Theorem: There is NO wait-free (or non-blocking) consensus

algorithm for 2 processes, using only atomic registers.
There is NO wait-free or non-blocking implementation of

a queue for two processes or more, from atomic registers.

Compare & swap (CAS)
shared
register old new
CAS ( A, B, C )
if A=B then A:=C; return(true)

else return(false)
Supported by Sun, Intel, AMD, …

CAS: The ABA problem
shared
register old new
local:= A
CAS ( A, B, C ) CAS (A, local, 100)
local: value tag

if A=B then A:=C
A: value tag
CAS (A, local, <100,local.tag+1> )
In the following, we will ignore the ABA problem and assume that it
is resolved
Chapter 4 using additional tag fields.
Synchronization Algorithms and Concurrent Programming 50
Section 4.5
A Non-blocking Queue
Ø The algorithm is due to M. M. Michael and M.

L. Scott (1996).
Ø The algorithm is included in the Standard Java

Concurrency Package.
Ø Every process must be prepared to encounter

a half-finished enqueue operation, and help to
finish it.
Ø Invariant: The tail refers to either the last

node or to the node just before the last node.

lnode tail head
Enqueue
repeat
ltail ç tail true
false
lnext ltail done
lnext ç ltail.next
if ltail = tail then
if lnext = NULL then
if CAS (ltail.next, lnext, lnode) then done ç true fi
else CAS (tail, ltail, lnext) fi fi
until done = true try to link lnode to
the end
enqueue failed: tryof
tothe queue
swing tail to
CAS (tail, ltail, lnode) a node
enqueue that
done: trywas inserted
to swing tailby
to some
the inserted node.
other process

lnode tail head
Enqueue
repeat
ltail ç tail true
false
lnext ltail done
lnext ç ltail.next
if lnext = NULL then
if CAS (ltail.next, lnext, node) then done ç true fi
until done = true
Question: Would the enqueue
CAS (tail, ltail, lnode) operation be correct if the last
line is omitted?
Chapter 4
Answer: Yes, but…
53
lnode tail head
Enqueue
repeat
ltail ç tail true
false
lnext ltail done
lnext ç ltail.next
Question: Would the enqueue
operation be correct if this line
if lnext = NULL then is omitted?
if CAS (ltail.next, lnext, node) then done ç true fi
until done = true No, when it is assumed that cells can
be released and then reallocated.
CAS (tail, ltail, lnode) Otherwise, yes.

tail head
Dequeue
repeat lvalue
lhead ç head
true
false
ltail ç tail
lnext ltail done lhead
lnext ç lhead.next
if lhead = head then
if lhead = ltail then /* empty or tail behind?
if lnext = NULL then return(NULL) fi ; /* empty
CAS (tail, ltail, lnext) /* try to advance tail
else lvalue ç lnext.value /* no need to deal with tail
if CAS (head, lhead, lnext) then done ç true fi fi fi
until done = true
free(lhead) ; return(lvalue)
tail head
Dequeue
repeat lvalue
lhead ç head
true
false
ltail ç tail
lnext ltail done lhead
lnext ç lhead.next
Question: Would the dequeue operation
if lhead = head then be correct if this line is omitted?
if lhead = ltail then /* empty or tail behind?
if lnext = NULL then return(NULL) fi ; /* empty
CAS (tail, ltail, lnext) /* try to advance tail
else lvalue ç lnext.value /* no need to deal with tail
if CAS (head, lhead, lnext) then done ç true fi fi fi
until done = true
free(lhead) ; return(lvalue)
A note about
Memory Barriers

Chapter 4
Example: Using Flags
x and y : atomic bits, initially 0
x 0 y 0
Process A Process B
write.x(1) write.y(1)
read.y read.x
Q: Is it possible that both

processes read the value 0 ?

x 0 y 0
Process A Process B
read.y read.x
Fact: Many hardware architectures do

not support sequential consistency
because they think it is too strong L

x 0 y 0
Process A Process B
read.y read.x
Solution: Memory barriers

Binary Semaphores
Section 4.6

Chapter 4
Section 4.6
Binary Semaphores
Operations down(S)
m if S > 0, then S = 0 otherwise,
down(S)
1 the process is blocked until the
up(S)
value becomes greater than 0.
m Testing and decrementing the
Types
semaphore are executed
unfair semaphore atomically without interruption.
weak semaphore
strong semaphore up(S)
m S=1

Semaphores
Deadlock-free Mutual exclusion Algorithm
down(S)
critical section 1
up(S)

Semaphores
Constant Space Starvation-free Algorithm
Weak semaphores
queue
S.0 1
10
S.1 1
Shared bits
queue 0
S.0 10
1 S.1 0
1 empty false
local bits
doorkeeper empty myqueue

true
false otherqueue
critical
section
Semaphores
Weak semaphores
queue
S.0 1
1
S.1 1
Shared bits
queue 0
S.0 0
1 S.1 0 empty false
local bits
empty myqueue
true
false otherqueue
critical
section
Semaphores
Weak semaphores
queue
S.0 1
1
S.1 1
Shared bits
queue 0
S.0 1 S.1 0 empty false
local bits
empty myqueue
true
false otherqueue
critical
section
Semaphores
Weak semaphores
queue
S.0 1
1
S.1 1
Shared bits
queue 0
S.0 10
1 S.1 0 empty false
local bits
empty myqueue
true
false otherqueue
critical
section
Semaphores
Weak semaphores
queue
S.0 1
1
S.1 1
Shared bits
queue 0
S.0 1 S.1 0 empty false
local bits
empty myqueue
true
false otherqueue
critical
section
Semaphores
Weak semaphores
queue
S.0 1
1
S.1 1
Shared bits
queue 0
S.0 10
1 S.1 0 empty false
local bits
empty myqueue
true
false otherqueue
critical
section
Semaphores
Weak semaphores
queue
S.0 1
1
S.1 1
Shared bits
queue 0
S.0 0
1 S.1 10 empty false
local bits
empty myqueue
true otherqueue
critical
section
Semaphores
myeuque = queue Weak semaphores
down (S.myqueue)
if queue = myqueue then S.0 1
otherqueue = 1 – myqueue S.1 1
down(S.otherqueue)
queue = otherqueue Shared bits
repeat
empty = true queue 0
up(S.myqueue) empty false
down(S.myqueue)
until empty local bits
critical section
up(S.otherqueue) myqueue
else empty = false otherqueue
critical section fi
up(S.myqueue)
Semaphores
Question: Is it possible
Yes
queue that some process will
be in its CS, while no
10 process is a doorkeeper?
S.0 10
1 S.1 1
empty
true
false
critical
section
Properties of the Algorithm
q Satisfies mutex & deadlock-freedom.

q Satisfies 2-bounded-waiting: A process can enter its
CS ahead of anoter only twice.
q There is no need to know the # of processes
q System response time is a constant
q Two weak semaphores and two atomic bits are used.

The End


Chapter4 ConcurrentDataStructures

Uploaded by

Copyright:

Available Formats

You might also like

Chapter4 ConcurrentDataStructures

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Chapter4 ConcurrentDataStructures

Uploaded by

Copyright:

Available Formats

Synchronization Algorithms

and Concurrent Programming

Version: June 2014

A note on the use of these power-point slides:

Thanks and enjoy!

To get the most updated version of these slides go to:

4.1 Synchronization Primitives

Chapter 4 Synchronization Algorithms and Concurrent Programming 3

data structure data structure

Chapter 4 Synchronization Algorithms and Concurrent Programming 4

data structure data structure

• simple programming model • resilient to failures, etc.

Synchronization Algorithms and Concurrent Programming 6

data structure data structure

Chapter 4 Synchronization Algorithms and Concurrent Programming 8

Chapter 4 Synchronization Algorithms and Concurrent Programming 9

Chapter 4 Synchronization Algorithms and Concurrent Programming 10

Done Done Done Done

Chapter 4 Synchronization Algorithms and Concurrent Programming 11

Obstruction-freedom Non-blocking Wait-freedom

• too weak progress condition • strong enough • strong/desirable

Chapter 4 Synchronization Algorithms and Concurrent Programming 12

Synchronization Algorithms and Concurrent Programming

Ø Formal specification of concurrent objects

Ø Help to understand possible behaviors of an implementation

Ø Clarifies what optimizations are acceptable

Synchronization Algorithms and Concurrent Programming

Synchronization Algorithms and Concurrent Programming

Is this execution reasonable? Yes

Synchronization Algorithms and Concurrent Programming

Is this execution reasonable? ???

Synchronization Algorithms and Concurrent Programming

Is this execution reasonable? Yes

Synchronization Algorithms and Concurrent Programming

Is this execution reasonable? Yes

Synchronization Algorithms and Concurrent Programming

Synchronization Algorithms and Concurrent Programming

Each concurrent execution is Each sequential execution

Ø Each operation should appear to “take effect” instantaneously

Ø Object is correct if this “sequential” behavior is correct

Synchronization Algorithms and Concurrent Programming

Ø Permits programmers to specify and reason about concurrent

Ø provides the illusion that each operation applied by concurrent

Synchronization Algorithms and Concurrent Programming

Invocation Linearization Response

Synchronization Algorithms and Concurrent Programming

Is this execution linearizable? Yes

Synchronization Algorithms and Concurrent Programming

Is this execution linearizable? Yes X 2

Synchronization Algorithms and Concurrent Programming

Is this execution linearizable? No, BUT …

Synchronization Algorithms and Concurrent Programming

Is this execution linearizable? No, BUT …

Synchronization Algorithms and Concurrent Programming

Synchronization Algorithms and Concurrent Programming

Ø No need to preserve real time order

Ø Each operation should “take effect” instantaneously

Ø Object is correct if this “sequential” behavior is