Algorithmics

CT065-3.5-3

Assessing Algorithmic
Performance And Correctness
Level 3 – Computing (Software Engineering)
 Topic and Structure of the
Lesson
1. Running-time of algorithms
2. Kinds of performance analyses
3. Asymptotic Analysis
4. Analysis of Insertion and Merge Sort
5. Correctness of algorithms
6. Group Exercise
- Analysis of Selection Sort
- Q&A on correctness of algorithms
Module Code and Module Title Title of Slides Slide 2 (of 26)
 Running Time of Algorithms
• The running time of an algorithm can be
indicative of its efficient implementation (slide
14, Lecture 1)
• For example, the Merge Sort had a faster
running time as n increased than Insertion and
Selection sort.
• Running time depends on the input: an already
sorted sequence is easier to sort!
• We generally seek upper bounds on the running
time, because we want a guarantee that the
algorithm will run within a certain time frame.

Module Code and Module Title Title of Slides Slide 5 (of 26)
 Learning Outcomes

• Apply O, Θ, Ω notation to the analysis of

algorithms

• Prove the correctness of an algorithm

Module Code and Module Title Title of Slides Slide 3 (of 26)
 Key Terms

1. Asymptote
2. Big O(O)
3. Theta(Θ)
4. Omega(Ω)

Module Code and Module Title Title of Slides Slide 4 (of 26)
 Kinds of Performance Analyses
Worse-case: (usually)
• T(n) = maximum time of algorithm on any input
of size n.
Average-case: (sometimes)
• T(n) = expected time of algorithm over all
inputs of size n.
• Need assumption of statistical distribution of
inputs
Best-case: (bogus)
• Cheat with a slow algorithm that works fast on
some input.

Module Code and Module Title Title of Slides Slide 6 (of 26)
 Asymptotic Analysis

• What if we looked at Insertion Sort’s worst-

case time?
– It would vary based on the speed of our
computer:
• relative speed (on the same machine)
• Absolute speed (on different machines)
– If we ignore machine-dependency, we can
look at the growth of T(n) as n ∞
• This is called Asymptotic Analysis
Module Code and Module Title Title of Slides Slide 7 (of 26)
 Asymptotic Notation
• O – notation (upper bounds)
– f(n) = O(g(n)) if there exist constants c > 0,
n0 > 0 such that 0 ≤ f(n) ≤ cg(n) for all n ≥ n0.
– Example:2n2 = O(n3) (c = 1, n0 = 2)
• Ω – notation (lower bounds)
– Ω(g(n))= { f(n) : there exist constants c > 0,
n0 > 0 such that 0 ≤ cg(n) ≤ f(n) for all n ≥ n0 }
– Example: √n = Ω(lg n) (c = 1, n0 = 16)
• Θ – notation (tight bounds)
– Θ(g(n)) = O(g(n)) ∩ Ω(g(n))
– Example: ½ n2 – 2n = Θ (n2)
Module Code and Module Title Title of Slides Slide 8 (of 26)
 Asymptotic Performance
When n gets large enough, a Θ(n2) algorithm
always beats a Θ(n3) algorithm.
• We should not ignore
asymptotically slower
algorithms, however.
• Real-world design
situations often call for
a balancing of
objectives
• Asymptotic analysis
is a useful tool to help
to structure our
thinking.
Module Code and Module Title Title of Slides Slide 9 (of 26)
 Correctness of Algorithms

Is “Sort the sequence {8, 1, 5, 3, 4}” a problem?

– No…it is a problem instance. The problem is
sorting.
– A problem is a collection of problem instances
– An algorithm is correct for a problem instance if it
produces the correct answer for the instance
– An algorithm is correct for a problem if it is
correct for all instances of that problem.
– We want algorithms that are correct for
problems, not just problem instances.

Module Code and Module Title Title of Slides Slide 16 (of 26)
 Correctness of Algorithms
How do we prove that an algorithm is correct?

• Induction
• Loop invariant
• Proof by contradiction

We can use a combination of these to prove

correctness. For example, to prove Merge Sort is
correct, we could use induction (for the recursion
portion) and loop invariant (for the loop portion).

Module Code and Module Title Title of Slides Slide 17 (of 26)
 Induction

When can induction be used?

- Recursion
Module Code and Module Title Title of Slides Slide 18 (of 26)
 Loop Invariant

When can loop invariant be used?

- Iteration
Module Code and Module Title Title of Slides Slide 19 (of 26)
 Proof by Contradiction

Module Code and Module Title Title of Slides Slide 20 (of 26)
 Correctness of Insertion Sort
for j ← 2 to n do
// Invariant 1: A[1..j-1] is a sorted
// permutation of the original A[1..j-1]
key ← A[j]
i ← j-1
while (i > 0 and A[i] > key) do
// Invariant 2: A[i .. j] are each >= key
A[i+1] ← A[i]
i ← i -1
A[i+1] ← key

Claim:
• At the start of each iteration of the for loop,
the subarray A[1..j −1] consists of the
elements originally in A[1..j −1] and in
sorted order.
Module Code and Module Title Title of Slides Slide 21 (of 26)
 Correctness of Insertion Sort
• Proof of Claim:
– initialisation: j = 2
– maintenance: j  j +1
– termination: j = n+1

• Loop invariant (LI) General Steps:

– Initialization: does LI hold 1st time through?
– Maintenance: if LI holds one time, does LI hold
the next?
– Termination #1: upon completion, LI implies
correctness?
– Termination #2: does loop terminate?

Module Code and Module Title Title of Slides Slide 22 (of 26)
 Correctness of Insertion Sort
Insert sort LI:
At start of for loop, keys initially in A[1..j−1] are in
A[1..j−1] and sorted.

• Initialization: A[1..1] is trivially sorted

• Maintenance: none from A[1..j] moves beyond j;
sorted
• Termination #1: upon completion, j = n+1 and by
LI A[1..n] is sorted
• Termination #2: for loop counter j increases by 1
at a time, and no change inside the loop
Module Code and Module Title Title of Slides Slide 23 (of 26)
 Quick Review and Practice

• What do we mean by correctness of an

algorithm?
• What methods of proving correctness are
available?

Develop a proof of correctness for the

Selection Sort. Decide what method you will use,
and go through the steps of the proof as we did for
Insertion Sort.
Module Code and Module Title Title of Slides Slide 24 (of 26)
 Q&A

Any Questions?

Module Code and Module Title Title of Slides Slide 25 (of 26)
 Summary

1. Running-time of algorithms
2. Kinds of performance analyses
3. Asymptotic Analysis
4. Analysis of Insertion and Merge Sort

Module Code and Module Title Title of Slides

 Next Lesson

• Sorting techniques and analysis

– Computational Complexity
– Order of execution
– Best approach

Module Code and Module Title Title of Slides Slide 26 (of 26)