Book:, By:: Introduction To Algorithms

DAA
Book: Introduction to Algorithms, by:
Thomas H. Cormen
Charles E. Leiserson
Ronald L. Rivest
Clifford Stein
Shashank Dwivedi 1
Lecture Contents (objectives)
 Algorithm Definition
 Insertion sort
 Analysis of insertion sort
 Example of insertion sort
 (Best, Worst and Average) case analysis
 Merge sort
 The divide-and-conquer approach
 Analyzing merge sort
 Example of merge sort
 Recursion tree
Computer Sciences Department 2

Algorithms
 Informally, an algorithm is any well-defined

computational procedure that takes some value, or
set of values, as input and produces some value, or
set of values, as output.
 An algorithm is thus a sequence of computational

steps that transform the input into the output.

Design and Analysis of Algorithms
• Analysis: predict the cost of an algorithm in
terms of resources and performance
• Design: design algorithms which minimize the

cost

 An algorithm is said to be correct if, for every input instance,
it halts with the correct output

Algorithms as a technology
 Would you have any reason to study algorithms? YES.

 If computers were infinitely fast, any correct method for
solving a problem would do.
 Computers may be fast, but:
- memory may be cheap, but it is not free. Computing
time is therefore a bounded resource, and so is space in
memory. These resources should be used wisely, and
algorithms that are efficient in terms of time or space
will help you do so.
Very important
 Which computer/ algorithm is

faster?

Getting Started

Running time
• The running time depends on the input: an

already sorted sequence is easier to sort.
• Major Simplifying Convention:
Parameterize the running time by the size of
the input, since short sequences are easier to
sort than long ones.
TA(n) = time of A on length n inputs
• Generally, seek upper bounds on the
running time, to have a guarantee of
performance.
Kinds of analyses
Worst-case: (usually)
• T(n) = maximum time of algorithm on any input
of size n. (upper bound on the running time)
Average-case: (sometimes)
• T(n) = expected time of algorithm over all inputs
of size n.
Best-case:
• T(n) = minimum time of algorithm ((fastest
time to complete, with optimal inputs chosen)
(lower bound on the running time))
Worst-case and average-case analysis
 The best case, in which the input array was already

sorted, and the worst case, in which the input array
was reverse sorted.
 The worst-case running time of an algorithm is an
upper bound on the running time for any input.
How long does it take to determine where in sub-array

A[1 . . j − 1] to insert element A[ j ]?
Average case
 Suppose that we randomly choose n numbers and apply

insertion sort.
How long does it take to determine where in sub-array A[1 . .
j − 1] to insert element A[ j ]?
 On average, half the elements in A[1 . . j − 1] are less than

A[ j ], and half the elements are greater.
 On average, therefore, we check half of the sub-array
A[1 . . j − 1], so t j = j/2.
Start
Insertion sort
One move / comparing / insert it into the correct position
6 5 3 1 8 7 2 4
The numbers that we wish to sort are also

known as the keys
Insertion sort, is an efficient algorithm for
sorting a small number of elements.
Insertion sort (cont’d)

Insertion sort (cont’d) “Pseudo-code conventions”
1 i j n
A:
Computer Sciences Department
sorted
17 key Length[A]=n
Analyzing algorithms
 predicting the resources that the algorithm requires.

 random-access machine (RAM)- Instructions are
(executed one after another, with no concurrent
operations)
 The data types in the RAM model are integer and
floating point and limit on the size of each word of data.
 Is exponentiation a constant time instruction? “shift
left” instruction.

Analysis of insertion sort
 The time taken by the INSERTION-SORT procedure depends
on the input.
 INSERTION SORT can take different amounts of time:
- to sort two input sequences of the same size
depending on how nearly sorted they already
are.
- to sort thousand numbers or three numbers.
 The running time of an algorithm on a particular input is the

number of primitive operations or “steps” executed.
Analysis of insertion sort (cont’d)
Best case
Worst case
Analysis of insertion sort (cont’d)

t i on
lana
Exp
a b
c =an2+bn-c
Example of insertion sort
8 1 4 9 3 7

8 1 4 9 3 7

8 1 4 9 3 7
1 8 4 9 3 7

8 1 4 9 3 7
1 8 4 9 3 7

8 1 4 9 3 7
1 8 4 9 3 7
1 4 8 9 3 7

8 1 4 9 3 7
1 8 4 9 3 7
1 4 8 9 3 7

8 1 4 9 3 7
1 8 4 9 3 7
1 4 8 9 3 7
1 4 8 9 3 7

8 1 4 9 3 7
1 8 4 9 3 7
1 4 8 9 3 7
1 4 8 9 3 7

8 1 4 9 3 7
1 8 4 9 3 7
1 4 8 9 3 7
1 4 8 9 3 7
1 3 4 8 9 7

8 1 4 9 3 7
1 8 4 9 3 7
1 4 8 9 3 7
1 4 8 9 3 7
1 3 4 8 9 7

8 1 4 9 3 7
1 8 4 9 3 7
1 4 8 9 3 7
1 4 8 9 3 7
1 3 4 8 9 7
1 3 4 7 8 9
The divide-and-conquer approach
 Divide the n-element sequence to be sorted into two

subsequences of n/2 elements each.
 The merge sort algorithm closely follows the divide-and-
conquer paradigm.

MERGE-SORT
MERGE-SORT A[1 . . n]
1. If n = 1, done.
2. Recursively sort A[ 1 . . n/2 ]
and A[ n/2+1 . . n ] .
3. “Merge” the 2 sorted lists.

Analyzing merge sort
T(n) MERGE-SORT A[1 . . n]
Q(1) 1. If n = 1, done.
2T(n/2) 2. Recursively sort A[ 1 . . n/2 ]
and A[ n/2+1 . . n ] .
Q(n) 3. “Merge” the 2 sorted lists
Recurrence for merge sort

Q(1) if n = 1;
T(n) =
2T(n/2) + Q(n) if n > 1.
Merge sort
 MERGE(A, p, q, r), where A is an array and p, q, and r
are indices numbering elements of the array such
that p ≤ q < r.

Merging two sorted arrays
20 12
13 11
7 9
2 1

20 12
13 11
7 9
2 1

20 12 20 12
13 11 13 11
7 9 7 9
2 1 2

20 12 20 12
13 11 13 11
7 9 7 9
2 1 2
1 2

20 12 20 12 20 12
13 11 13 11 13 11
7 9 7 9 7 9
2 1 2
1 2

20 12 20 12 20 12
13 11 13 11 13 11
7 9 7 9 7 9
2 1 2
1 2 7

20 12 20 12 20 12 20 12
13 11 13 11 13 11 13 11
7 9 7 9 7 9 9
2 1 2
1 2 7

20 12 20 12 20 12 20 12
13 11 13 11 13 11 13 11
7 9 7 9 7 9 9
2 1 2
1 2 7 9

20 12 20 12 20 12 20 12 20 12
13 11 13 11 13 11 13 11 13 11
7 9 7 9 7 9 9
2 1 2
1 2 7 9

20 12 20 12 20 12 20 12 20 12
13 11 13 11 13 11 13 11 13 11
7 9 7 9 7 9 9
2 1 2
1 2 7 9 11

20 12 20 12 20 12 20 12 20 12 20 12
13 11 13 11 13 11 13 11 13 11 13
7 9 7 9 7 9 9
2 1 2
1 2 7 9 11

20 12 20 12 20 12 20 12 20 12 20 12
13 11 13 11 13 11 13 11 13 11 13
7 9 7 9 7 9 9
2 1 2
1 2 7 9 11 12

20 12 20 12 20 12 20 12 20 12 20 12
13 11 13 11 13 11 13 11 13 11 13
7 9 7 9 7 9 9
2 1 2
1 2 7 9 11 12
Time = Q(n) to merge a total of n elements (linear
time).

Merge sort - “Pseudo-code conventions”

Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.

Recursion tree

Recursion tree
T(n)

Recursion tree
cn
T(n/2) T(n/2)

Recursion tree
cn
cn/2 cn/2
T(n/4) T(n/4) T(n/4) T(n/4)

Recursion tree
cn
cn/2 cn/2
cn/4 cn/4 cn/4 cn/4

…
Q(1)

Recursion tree
cn
cn/2 cn/2
h = lg n cn/4 cn/4 cn/4 cn/4
…
Q(1)

Recursion tree
cn cn
cn/2 cn/2
…
Q(1)

Recursion tree
cn cn
cn/2 cn/2 cn
…
Q(1)

Recursion tree
cn cn
cn/2 cn/2 cn
h = lg n cn/4 cn/4 cn/4 cn/4 cn
…
…
Q(1)

Recursion tree
cn cn
cn/2 cn/2 cn
…
…
Q(1) #leaves = n Q(n)

Recursion tree
cn cn
cn/2 cn/2 cn
…
…
Q(1) #leaves = n Q(n)
Total = Q(n lg n)
Conclusions
• Q(n lg n) grows more slowly than Q(n2).
• Therefore, merge sort asymptotically

beats insertion sort in the worst case.
• In practice, merge sort beats insertion

sort for n > 30 or so.

Book:, By:: Introduction To Algorithms

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Book:, By:: Introduction To Algorithms

Uploaded by

Copyright:

Available Formats

DAA

Book: Introduction to Algorithms, by:

Computer Sciences Department 2

 Informally, an algorithm is any well-defined

 An algorithm is thus a sequence of computational

Computer Sciences Department 3

• Design: design algorithms which minimize the

Computer Sciences Department 4

Computer Sciences Department 5

 Would you have any reason to study algorithms? YES.

 Which computer/ algorithm is

Computer Sciences Department 7

Computer Sciences Department 10

• The running time depends on the input: an

 The best case, in which the input array was already

How long does it take to determine where in sub-array

 Suppose that we randomly choose n numbers and apply

 On average, half the elements in A[1 . . j − 1] are less than

The numbers that we wish to sort are also

Computer Sciences Department 16

 predicting the resources that the algorithm requires.

Computer Sciences Department 18

 The running time of an algorithm on a particular input is the

Computer Sciences Department 21

Computer Sciences Department 23

Computer Sciences Department 24

Computer Sciences Department 25

Computer Sciences Department 26

Computer Sciences Department 27

Computer Sciences Department 28

Computer Sciences Department 29

Computer Sciences Department 30

Computer Sciences Department 31

Computer Sciences Department 32

 Divide the n-element sequence to be sorted into two

Computer Sciences Department 34

Computer Sciences Department 36

Recurrence for merge sort

Computer Sciences Department 39

Computer Sciences Department 41

Computer Sciences Department 42

Computer Sciences Department 43

Computer Sciences Department 44

Computer Sciences Department 45

Computer Sciences Department 46

Computer Sciences Department 47

Computer Sciences Department 48

Computer Sciences Department 49

Computer Sciences Department 50

Computer Sciences Department 51

Computer Sciences Department 52

Computer Sciences Department 53

Computer Sciences Department 54

Computer Sciences Department 55

Computer Sciences Department 56

Computer Sciences Department 57

Computer Sciences Department 58

T(n/4) T(n/4) T(n/4) T(n/4)

Computer Sciences Department 59

cn/4 cn/4 cn/4 cn/4

Computer Sciences Department 60

Computer Sciences Department 61