‫بسم هللا الرحمن الرحيم‬

‫‪1‬‬
‫‪Prof. Ossama Ismail‬‬
 Introduction to Algorithms - II

Prof. Ossama Ismail

2
Prof. Ossama Ismail
 Introduction to Algorithms

Lecture 1-2
Problem Solving
&
Mathematical Tools
3
Prof. Ossama Ismail
 PROBLEM SOLVING

DEFINITION

“Creative problem solving is - looking at

the same thing as everyone else and thinking
something different.”

Adapted from a famous quote from a former Nobel prize winner,

Albert Szent-Gyorgyi.

4
Prof. Ossama Ismail
 OBJECTIVE: Learn how to solve problems

• PURPOSE:

• To develop the awareness and the skills

necessary to solve problems creatively.

5
Prof. Ossama Ismail
 How do we solve problems?

• We "just do"
• Guesswork-and-luck
• Trial-and-error
• Experience (possibly someone else's)
• "Scientifically"

6
Prof. Ossama Ismail
 The Problem-solving Process

Analysis
Problem
specification
Design

Algorithm
Implementation

Program

Compilation

Executable
(solution)
7
Prof. Ossama Ismail
 Creative Problem Solving

• The creative person uses information to form new

ideas.

• The real key to creative problem solving is what

you do with the knowledge.

• Creative problem solving requires an attitude that

allows you to search for new ideas and use your
knowledge and experience.

8
Prof. Ossama Ismail
 Problem Solving
Steps
– Goal Formulation
• limiting the objectives
– Problem Formulation
• deciding what actions and states to consider
– Search
• looking for the possible action sequence
– Execution
9
Prof. Ossama Ismail
 TIPS TO PRACTICAL PROBLEM
SOLVING

One GREAT tip for problem solving

is to NEVER be one DIMENSIONAL

about the whole process, creativity

in many DIFFERENT FORM will

guide you to the Answer

10
Prof. Ossama Ismail
 The Development of Problem Solving Skills

 The interpersonal problem solving skills can be

learned from experiences beginning in the family
and wherever the child interacts with others in
situations that give rise to interpersonal difficulties.

 How well the developing child learns these skills is

thought to reflect the extent to which the child’s
caregivers manifest these abilities themselves.

11
Prof. Ossama Ismail
 Example 1 : The 8-queens problem

Problem formulation
– State : any arrangement of 0 to 8 queens on board
– Operators: add a queen to any square
– Goal test: 8 queens on board, none attacked

12
Prof. Ossama Ismail
 Example 2 : Number of rectangles

How many rectangles appear in the figure below?

13
Prof. Ossama Ismail
 Example 2 : Number of rectangles – cont’d

• Ask yourself ??

• Did you resolve the question about whether to

include the squares in your count of rectangles?

• Did you understand that squares meet all the

criteria to be considered a rectangle?
• Did you recognize that there are many different
sizes of rectangles in the drawing?
14
Prof. Ossama Ismail
 Example 2 : Number of rectangles – cont’d

◼ Have you devised a way of counting the

rectangles you find?

◼ Do you find patterns in the number of

different-sized rectangles?

◼ Do you think about the concepts embedded

in the problem differently than before?
15
Prof. Ossama Ismail
 Example 2 : Number of rectangles – cont’d

Now try to solve these problems

16
Prof. Ossama Ismail
 Example 3 : Move 3 sticks

Given the following

configuration of sticks,
Move exactly 3 sticks
in such a way that the
remaining configuration
forms exactly 3 squares.

17
Prof. Ossama Ismail
 Example 4 :Water Jug Problem
Given a full 5-gallon jug
and an empty 2-gallon
Operation table
jug, the goal is to fill Name Con Transition Effect
the 2-gallon jug with d.
exactly one gallon of Empty5 – (x,y)→(0,y) Empty 5-
water. gal. jug

State = (x,y), where x is Empty2 – (x,y)→(x,0) Empty 2-

the number of gallons gal. jug
of water in the 5-gallon 2to5 x≤3 (x,2)→(x+2, Pour 2-gal.
jug and y is # of gallons 0) into 5-gal.
in the 2-gallon jug 5to2 x≥2 (x,0)→(x- Pour 5-gal.
Initial State = (5,0) 2,2) into 2-gal.

Goal State = (*,1), where 5to2part y < 2 (1,y)→(0,y+ Pour partial

* means any amount 1) 5-gal. into
2-gal. 18
Prof. Ossama Ismail
 Example 5 : Nine Dots Problem
How to connect all 9 dots arranged in a square
with only 4 straight lines?
Thinking outside the box

One of many solutions to

inetheDots
puzzleProblem
is to go beyond
the boundaries to link all
dots in 4 straight lines.

19
Prof. Ossama Ismail
 Example 5 : Nine Dots Problem –cont’d

The origami-like solution:

This is our favorite one! Reproduce
the puzzle on a square sheet of paper.
By ingeniously folding it, according
to the example below, it is possible to
align the 9 dots in order to connect
them together with a final pencil
stroke.

20
Prof. Ossama Ismail
 Example 5 : Nine Dots Problem –cont’d

Is there some general formula to find the minimum

number of lines for a given number of dots?

No. of Dots Required Straight Lines

3x3 (3 + 1) = 4
4x4 (4 + 2) = 6
5x5 (5 + 3) = 8
6x6 (6 + 4) = 10
… …
nxn (n + n - 2) = 2n - 2
21
Prof. Ossama Ismail
 Example Real-World Problems

Examples :
❖ Internet Searching (e.g. Google –Yahoo)
❖ Route finding
❖ Touring and travelling salesperson problems
❖ VLSI layout
❖ Robot navigation
❖ Assembly sequencing

22
Prof. Ossama Ismail
 Problem solving tools and techniques

• Really want to solve the problem.

• Have wide-ranging interests, and feed them.
• Be willing to entertain ideas and inspirations
from outside the box—not only “think outside
the box.” Learn from any and every source.
• Be willing to keep coming back to the problem
from different directions.

23
Prof. Ossama Ismail
 Creative problem solving process

• STEP 1. State what appears to be the problem.

The real problem may not surface until facts have been
gathered and analyzed. Therefore, start with what you
assume to be the problem, that can later be confirmed
or corrected.

24
Prof. Ossama Ismail
Creative problem solving process – cont’d

• STEP 2. Gather facts, feelings and opinions.

o What happened?
o Where, when and how did it occur?
o What is it’s size, scope, and severity?
o Who and what is affected?
o Likely to happen again?
o Need to be corrected?
o May need to assign priorities to critical elements.

25
Prof. Ossama Ismail
Creative problem solving process – cont’d

• STEP 3. Restate the problem.

The real facts help make this possible, and

provide supporting data.
The actual problem may, or may not be the
same as stated in Step 1.

26
Prof. Ossama Ismail
Creative problem solving process – cont’d

• STEP 4. Identify alternative solutions.

Generate ideas. Do not eliminate any possible

solutions until several have been discussed.

27
Prof. Ossama Ismail
Creative problem solving process – cont’d

• STEP 5. Evaluate alternatives.

o Which will provide the optimum solution?

o What are the risks?
o Are costs in keeping with the benefits?
o Will the solution create new problems?

28
Prof. Ossama Ismail
Creative problem solving process – cont’d

• STEP 6. Implement the decision!

o Who must be involved?

o To what extent?
o How, when and where?
o Who will the decision impact?
o What might go wrong?
o How will the results be reported and verified?

29
Prof. Ossama Ismail
Creative problem solving process – cont’d

• STEP 7. Evaluate the results.

o Test the solution against the desired results.

o Make revisions if necessary.

30
Prof. Ossama Ismail
 Mathematical Tools

31
Prof. Ossama Ismail
 Mathematical Tools
• Floor and ceiling
• Polynomial of degree n
• Factorials
– n! = 1 * 2 * 3 * … * n
– 0! = 1
• Exponentials
a0 = 1
a1 = a
a-1 = 1/a
(am)n = amn
(am)n = (an)m
aman = am+n
ex = 1 + x + x2/2! + x3/3! + …
32
Prof. Ossama Ismail
 Mathematical Tools cont’d

• Summations
• Logarithms
lg n = log2 n
ln n = loge n
lg kn = (lg n)k
lg lgn = lg(lg n)
a = b(logba)
logb(1/a) = -logba
logba = 1/logab
alogbn = nlogba
33
Prof. Ossama Ismail
 Mathematical Tools cont’d
• Terminology
Function Name
c constant
a1n + a0 linear
a2n2 + a1n + a0 quadratic
a3n3 + a2n2 + a1n + a0 cubic
cn exponential
logbn logarithmic
nlogbn log linear

where c, a0, a1, … an are constants 34

Prof. Ossama Ismail
 Mathematical Tools cont’d
• Logarithms
Log (xy) = logx + log y
Log (x/y) = log x – log x
lgn = log2n
lnn = logen
Lgkn = (lgn)k
lglgn = lg(lg n)
a = b(logba)
logb(1/a) = -logba
logba = 1/logab
alogbn = nlogba
35
Prof. Ossama Ismail
 Algorithm Analysis

• Run-time is expressed as a function f(n),

where n is the size of the input

• How do we compute it?

– Each primitive statement is counted once
– A for loop that iterates k times will execute
(k + 1) + k * (execution time of the loop body)

36
Prof. Ossama Ismail
 Algorithm Analysis cont’d
Algorithm Sum(A, n):
input: The list A of integers
output: The sum of the integers in A
count
1. sum  0; 1
2. for i = 1 to n do n+1
3. sum  sum + A[i] n
4. return sum; 1

Running time ::

T(n) = 1 + (n + 1) + n + 1
= 2n + 3
= a linear function or takes “linear time” = O(n) 37
Prof. Ossama Ismail
 Algorithm Analysis cont’d
• Nested for loops
count
1. for i =1 to n do // n + 1
2. for j =1 to n do // (n )(n+1)
3. k  k + 1; // (n)(n)

• Running time ::

T(n) = (n + 1) + (n) (n + 1) +(n)*(n)

= 2n2 + 2n + 2
= a quadratic function
= O(n2) 38
Prof. Ossama Ismail
 Algorithm Analysis cont’d
count
1. for i = 1 to n step i  i * 2 do // (k+1 steps where n = 2k-1
2. for j = 1 to n do // k*(n+1)
3. k  k + (i– j); // k*n

• At the mth iteration, i is 2m-1

• If 2k-1 < n <= 2k, then the outer for loop executes k
times (n = 2k-1 → k = log n)
• Inner loop executes (n + 1) times
• Running time ::
• T(n) = k+1 + k*(n+1) + k*n
T(n) = log n * (2n + 2) = 2n log n + 2 log 2
= “n log n” = O(n log n).
Prof. Ossama Ismail
39
 Algorithm Analysis cont’d

1. for i = 1 to n step i  i * 2 do // ( k+1 steps where k = 2k-1)

2. for j = 1 to i do
3. m <- m + (i– j);

Running time:

The inner loop executes

1 + 2 + 4 + … + 2k-1 times for the k – 1 iterations of the
outer loop
T(n) = O(n )
what is the sum of 1+2 + 4 + * + … + 2k-1
40
Prof. Ossama Ismail
 Algorithm Analysis cont’d
count
1. for i = 1 to n do // n+1
2. for j =1 to n do // n*(n+1)
3. for k = 1 to n do // n*n*(n+1)
m <- m + 1; // n*n*n
Running time:

T(n) = n+1 + n(n+1) + n*n*(n + 1) + n*n*n )

= 2 n3 + 2 n2 + 2 n +1
= O(n3)
= “cubic function”
41
Prof. Ossama Ismail
 Algorithm Analysis cont’d

for(j = 1; j <= K; ++j)

for(s = 1; s <= K; ++s)
for(l = 1; l <= M; ++l)
foo();
for(j = 1; j <= N; ++j) O(K2 M + N M2)
for(s = 1; s <= M; ++s)
for(l = 1; l <= M; ++l)
foo();
42
Prof. Ossama Ismail
 Asymptotic Performance

• We will be interested in the behavior of the run time

function as n gets very large

• Also, we will be interested in the nature of the input

– Worst-case: input(s) where it takes the longest

– Average-case: how long it takes on most inputs
– Best-case: input(s) where it runs the fastest

43
Prof. Ossama Ismail
 Example:
Search Algorithms

44
Prof. Ossama Ismail
 Search Algorithms

Whenever large amounts of data

need to be accessed quickly, search
algorithms are considered.

45
Prof. Ossama Ismail
 Search Algorithms – cont’d
Many interesting problems can be solved using search algorithms: A
A search problem is defined by:
❖ A search space: –
▪ The set of objects among which we search for the solution
Examples: n-queens configuration, or routes between cities.

❖ A goal condition : –
▪ Characteristics of the object we want to find in the search
space? –
Example: Path between cities A and B

46
Prof. Ossama Ismail
 Search Algorithms – cont’d
Types of searching Algorithms :

1. Linear (Serial) search

2. Binary search
3. Jump search
4. Interpolation search
5. Exponential search
6. Sub-list search (search a linked list in another list)
7. Fibonacci search
8. The ubiquitous binary search
9. Recursive program to linearly search an element in a give
array
10. Recursive function to do substring search
47
Prof. Ossama Ismail
 Search Algorithms – cont’d
The Simplest Search: Linear Serial

• Items are stored in an array or list.

• To search for an item x:
– Start at the beginning of the list
– Compare the current item to x
– If unequal, proceed to next item

48
Prof. Ossama Ismail
 Search Algorithms – cont’d
The Simplest Search: Linear Serial

49
Prof. Ossama Ismail
 Search Algorithms – cont’d
Pseudo code for Serial Search
// Find x in an array A of length n
int i=1
boolean found = false
while ((i < n) && !found) {
if (x == A[i])
found = true
else i++
} >>>> print -- 50
Prof. Ossama Ismail
 Search Algorithms – cont’d
Analysis for Serial Search
• Best case: Requires one array access: Θ(1)
• Worst case: Requires n array accesses: Θ(n)
• Average case:
To access an item, assuming position is random
(assume uniform search):

(1+2+3+...+n)/n = n(n+1)/2n = (n+1)/2 = Θ(n)

51
Prof. Ossama Ismail
 Search Algorithms – cont’d
Binary Search
Binary search, is known as
▪ half-interval search,
▪ logarithmic search, or
▪ binarychop,
It is a search algorithm that finds the position of a target
value within a sorted array. Binary search compares the
target value to the middle element of the array.

• This method is “fast enough” and based on the

classical divide-and-conquer algorithm.
52
Prof. Ossama Ismail
 Search Algorithms – cont’d
Binary Search

53
Prof. Ossama Ismail
 Search Algorithms – cont’d
Binary Search

54
Prof. Ossama Ismail
 Search Algorithms – cont’d
// Pseudo code for Serial Search

BinarySearch( sortedArray[], first, last, key)

{ // sortedArray in array of sorted (ascending) values.
// first, last in lower and upper subscript bounds & key value to search for.
// returns: index of key, or -insertion_position -1 if key is not in the array.
count
if (first <= last) 1
{ mid = (first + last) / 2; // compute mid point. 1
if (key == sortedArray[mid]) return mid; // found it. 1
else if (key < sortedArray[mid]) 1
// recursively call BinarySearch for the lower part
BinarySearch(sortedArray, first, mid-1, key); T(n/2)
else
// recursively call BinarySearch for the upper part
BinarySearch(sortedArray, mid+1, last, key); } T(n/2)
}
return -(first + 1); // failed to find key 1
} 55
Prof. Ossama Ismail
 Search Algorithms – cont’d
Search for 50 in a list of 11-elements
4 5 11 23 31 43 48 50 61 66 70

4 5 11 23 31 43 48 50 61 66 70

4 5 11 23 31 43 48 50 61 66 70

4 5 11 23 31 43 48 50 61 66 70

4 5 11 23 31 43 48 50 61 66 70

4 5 11 23 31 43 48 50 61 66 70

4 5 11 23 31 43 48 50 61 66 70 56
Prof. Ossama Ismail
 Search Algorithms – cont’d
Analysis for Binary Search
1 for n = 1
T(n) = {
O(1) + T(n/2) for n > 1
Above equation is example of recurrence relation, this can be
solved by direct Substitution.
T(n) = T(n/2) +1
= T(n/22)+1+1
= T(n/23)+1+1+1
……
= T(n/2k) + k = log n // n = n/2k
T(n) = O(log n) --- order of log(n) 57
Prof. Ossama Ismail
 Search Algorithms – cont’d
Search for 50 in a list of 11-elements

58
Prof. Ossama Ismail
 Run-time Growth Rate
• We will be concerned about the growth rate of the
run-time function
– As n gets very large, what portion of the run-time
equation is dominant?
– See the plots in the following slide
• n+3
– The linear portion, or n, is dominant
• n2 + 3n + 1
– The quadratic portion, or n2, is dominant

59
Prof. Ossama Ismail
 Growth Rates of Functions
250

f(n) = n
f(n) = log(n)
f(n) = n log(n)
f(n) = n^2
f(n) = n^3
f(n) = 2^n

0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Why we care?
Because We worry about the speed of our algorithms for large input sizes “n”
60
Prof. Ossama Ismail
 Growth Rates of Functions cont’d

n=1 n=2 n=4 n=8 n=16 n=32

O(1) 1 1 1 1 1 1
O(logn) 0 1 2 3 4 5
O(n) 1 2 4 8 16 32
O(nlogn) 0 2 8 24 64 160
O(n2) 1 4 16 64 256 1024
O(n3), 1 8 64 512 4096 32768
O(2n) 2 4 16 235 65536 4294967296

Prof. Ossama Ismail

 Dominant Growth Rate

• Drop the lower order terms

• Drop the constant factors
• If two functions are multiplied, then take their
product

• If f(n) is a polynomial of degree d, then the

dominant portion is nd
• n2 + logn
– n2 is the dominant function

62
Prof. Ossama Ismail
 Big Oh Notation
• We will be concerned a lot about the
worst-case run-time (or upper bound)
– What is the dominant growth rate in the worst-
case?
• f(n) = O(g(n)),
– if there exists c and n0 > 0 such that
f(n) <= cg(n)
– f(n) grows at most as fast as g(n)

63
Prof. Ossama Ismail
 Big Oh Notation

• Most common notation used is "big O" notation

• Big “O” gives us a formal way of expressing
asymptotic upper bounds
• A way of bounding from above the growth of a
function
• Knowing where a function falls within the big-O
hierarchy allows us to compare it quickly with
other functions
• Gives us an idea of which algorithm has the best
time performance
• And yes, there is also a "little o" we'll see later.
64
Prof. Ossama Ismail
 Properties of Big Oh

• Any kth degree polynomial is O(nk)

• a * nk = O(nk) for any a > 0
• Big O is transitive
If f(n) = O(g(n)) and g(n) is O(h(n))
then f(n) = O(h(n))
• logan = O(logbn) for any a, b > 1
We do not care, asymptotically, what base we
take our logarithms to
(In a few cases, it does matter.)

65
Prof. Ossama Ismail
 Properties of Big Oh Cont’d

• Big O of a sum of functions is big O of the

largest function
How do you know which one is the largest?
The one that all the others are big O of
One consequence of this is,
if f(n) = O(h(n)) and g(n) is O(h(n)),
then f(n) + g(n) = O(h(n))
• f(n) = O(g(n)) is true if limn->f(n)/g(n) = constant

66
Prof. Ossama Ismail
 Big Oh Notation cont’d

➢ 2n + 10 = O(n)
➢ 7n – 2 = O(n)
➢ n2 + 5n – 3 = O(n2)
➢ 3n3 + 20n2 + 5 = O(n3)
➢ 3logn + log logn = O(logn)

67
Prof. Ossama Ismail
 Tight Upper Bound

• n2 is not in O(n)
To prove this assume that n2 <= cn
Then n <= c
==> Does not hold since “n” grows infinitely

• We want the “tightest” upper bound

– We found that insertion sort takes O(n2)
– It is also O(n3). Why?
– It is not O(n). Why? 68
Prof. Ossama Ismail
 Some complexity classes …
Constant O(1)
Logarithmic O(log n)
Linear O(n)
Quadratic O(n2)
Cubic O(n3)
Polynomial O(np)
Exponential O(an)
69
Prof. Ossama Ismail
 Don’t be confused …

• We typically say,
• f(n) is O(g(n)) or f(n) = O(g(n))
• But O(g(n)) is really a set of functions.
• It might be more clear to say,
• f(n) є O(g(n))
• But I don’t make the rules.
• Crystal clear: “f(n) is order (g(n))”

70
Prof. Ossama Ismail
 Intuitively …
• To say f(n) is O(g(n)) is to say that
• f(n) is “less than or equal to” g(n)
• We also have

Θ(g(n)) Big Theta “equal to”

Ω(g(n)) Big Omega “greater than or equal to”

o(g(n)) Little o “strictly less than”
ω(g(n)) Little omega “strictly greater than”

71
Prof. Ossama Ismail
 Big Omega Notation

• Best case run-time or lower bound

• f(n) = Omega(g(n)),
– if there exists c and n0 > 0 such that
f(n) >= cg(n)
– f(n) grows at least as fast as g(n)

72
Prof. Ossama Ismail
 Big-Omega and Big-Theta

• Ω is just like O except that f(n) ≥ cg(n);

• f(n) is O(g(n))  g(n) is Ω(f(n))

• Θ is both O and Ω (and the constants need not

match);
• f(n) is O(g(n))  f(n) is Ω(g(n))  f(n) is Θ(g(n))

73
Prof. Ossama Ismail
 Big Theta Notation

• A tight bound to run-time

• f(n) = Theta(g(n)),
– if there exists c0, c1, and n0 > 0 such that
c0g(n) <= f(n) <= c1g(n)
– f(n) grows at the same rate as g(n)

74
Prof. Ossama Ismail
 Big Theta Notation cont’d

f(n) = Theta(g(n)) iff

f(n) = O(g(n)) AND f(n) = Omega(g(n))

– Simple example
for i <- 1 to n do
for j <- 1 to n do
k <- k + 1;
O(n2) and Omega(n2) → Theta(n2)
– Insertion sort
• O(n2) and Omega(n) → Theta(n2)

75
Prof. Ossama Ismail
 Mappings for n 2

Ω (n2 )

Θ(n2)

O(n2 )

76
Prof. Ossama Ismail
 Comparing Growth Rates

• f(n) = O(g(n)) iff

lim (f(n)/g(n)) <= c

• f(n) = Omega(g(n)) iff

lim (f(n)/g(n)) >= c

• f(n) = Theta(g(n)) iff

c1 <= lim (f(n)/g(n)) <= c2

• Where c, c1, c2 > 0 are constants and n approaches infinity

77
Prof. Ossama Ismail
 Does it matter?
Let n = 1,000, and 1 ms / operation.

n = 1000, 1 ms/op
n 1 second
n log2 n 10 seconds
n2 17 minutes
n3 12 days
n4 32 years
n10 3.17  1019 years
2n 1.07  10301 years
78
Prof. Ossama Ismail
 RECURRENCE RELATIONS

79
Prof. Ossama Ismail
 Recurrence Relations
• A Recurrence is an equation or inequality that
describes a function in terms of its value on
smaller inputs.
=c for n = 1
T(n) = {
= 2 T(n/n) + c n for n > 1

• Special techniques are required to analyze the

space and time required
80
Prof. Ossama Ismail
 Solving Recurrences

• The substitution method

• The recursion method
• The master method

81
Prof. Ossama Ismail
 1. The substitution method
1. Guess the form of the solution.
2. Use mathematical induction to find the
constants and show that the solution works.

• Ex: T(n) = 2 T(n/2) + n. // Sorting

1. We guess that T(n) = O(n lg n)
2. T(n) ≤ 2(c n/2 lg(n/2)) + n ≤ cn lg(n/2) + n
= cn lg n - cn lg 2 + n = cn lg n - cn + n
≤ cn lg n
82
Prof. Ossama Ismail
 2. Recursion method
Sum all the per-level costs to determine the total
cost of all levels of the recursion

Ex: T(n) = 3T(n/4)+n

T(n) = n + 3 T(n/4)
= n + 3(n/4 + 3T(n/16))
= n + 3 n/4 + 3 (n/16 + 3T(n/64))
≤ n + 3 n/4 + 9 n/16 + …
= O(n2) 83
Prof. Ossama Ismail
 3. The master method
Master theorem: Let a ≥ 1 and b > 1 be constants,
let f(n) be a
function, and let T (n) be defined by
T(n) = aT(n/b) + f(n
Then T (n) can be bounded asymptotically as follows.
1.If T (n) =  (n logb a ) for some constant ε > 0, then T (n) = (n logb a )

2.If f (n) = (n logb a ) then T (n) = (n logb a lg n)

3.If f (n) = (n logb a +  ) for some constant ε > 0, and

if a f (n/b) ≤ c(n) for constant c <1 and all sufficiently
large n, then T (n) = Θ(f (n)).
84
Prof. Ossama Ismail
 Recurrence - Example 1

Binary Search

85
Prof. Ossama Ismail
 Example: Binary Search
BinarySearch( sortedArray[], first, last, key)
{ // sortedArray in array of sorted (ascending) values.
// first, last in lower and upper subscript bounds & key value to search for.
// returns: index of key, or -insertion_position -1 if key is not in the array.
count
if (first <= last) 1
{ mid = (first + last) / 2; // compute mid point. 1
if (key == sortedArray[mid]) return mid; // found it. 1
else if (key < sortedArray[mid]) 1
// recursively call BinarySearch for the lower part
BinarySearch(sortedArray, first, mid-1, key); T(n/2)
else
// recursively call BinarySearch for the upper part
BinarySearch(sortedArray, mid+1, last, key); } T(n/2)
}
return -(first + 1); // failed to find key 1
} 86
Prof. Ossama Ismail
 Recurrence -
Example 1:Binary Search Analysis
T(n)=O(1) + T(n/2)
T(1)=1
Above equation is example of recurrence relation, this can be
solved by by direct Substitution.
T(n) = T(n/2) +1
= T(n/22)+1+1
= T(n/23)+1+1+1
……
= T(n/2k) + k = log n // n = n/2k
T(n) = O(log n)
87
Prof. Ossama Ismail
 Recurrence - Example 2

Quick Sort

88
Prof. Ossama Ismail
EXAMPLE 2: QUICK SORT

Run time
T(n) = 2T(n/2) + O(n)
T(1)= O(1)

• In the above case the presence of function of T on both

sides of the equation signifies the presence of recurrence
relation
• (SUBSTITUTION MEATHOD used) The equations are
simplified to produce the final result:
……cntd
89
Prof. Ossama Ismail
EXAMPLE 2: QUICK SORT cont’d

T(n) = 2T(n/2) + O(n)

= 2(2(n/22) + (n/2)) + n
= 22 T(n/22) + n + n
= 22 (T(n/23)+ (n/22)) + n + n
= 23 T(n/23) + n + n + n
…
= 2k T(n/2k) +k n // n = 2k
= n log n // k = log n
90
Prof. Ossama Ismail
 Algorithms
• Types of Heuristic algorithms
1. ƒGreedy
2. Local search

• Types of Optimal algorithms

1. Exhaustive Search (ES)

2. Divide and Conquer (D&C)
3. Branch and Bound (B&B)
4. Dynamic Programming (DP)
91
Prof. Ossama Ismail
 Proof By Induction
• Claim: S(n) is true for all n >= k
• Basis:
▪ Show formula is true when n = k
• Inductive hypothesis:
▪ Assume formula is true for an arbitrary n
• Step:
▪ Show that formula is then true for n+1

92
Prof. Ossama Ismail
 Proof By Induction
Gaussian Closed Form
Prove 1 + 2 + 3 + … + n = n(n+1) / 2
• Basis:
▪ If n = 0, then 0 = 0(0+1) / 2 = 0
• Inductive hypothesis:
▪ Assume 1 + 2 + 3 + … + n = n(n+1) / 2
• Step (show true for n+1):
▪ 1 + 2 + … + n + n+1 = (1 + 2 + …+ n) + (n+1)
▪ = n(n+1)/2 + n+1 = [n(n+1) + 2(n+1)]/2
▪ = (n+1)(n+2)/2 = (n+1)(n+1 + 1) / 2
Prof. Ossama Ismail
93
 Proof By Induction
Geometric Closed Form
Prove a0 + a1 + … + an = (an+1 - 1)/(a - 1) for a ≠ 1
• Basis: show that a0 = (a0+1 - 1)/(a - 1)
a0 = 1 = (a1 - 1)/(a - 1)
• Inductive hypothesis:
▪ Assume a0 + a1 + … + an = (an+1 - 1)/(a - 1)
• Step (show true for n+1):
▪ a0 + a1 + … + an = (an+1 - 1)/(a - 1) + an+1
▪ = (an+1 - 1)/(a - 1) + an+1 = (an+1+1 - 1)/(a - 1)
94
Prof. Ossama Ismail
 Reading

Chapter two of Your Text Book

95
Prof. Ossama Ismail
 Summary

• Rules for Big-Oh

– if T(n) = O(c * f(n)) →T(n) = O(f(n))
– if T1(n) = O(f(n)) and T2(n) = O(g(n)) →
T1(n) + T2(n) = O(max(f(n), g(n)))
– if T1(n) = O(f(n)) and T2(n) = O(g(n)) →
T1(n) * T2(n) = O(f(n) * g(n))
– if T(n) = aknk + ak-1nk-1 + … a1n + a0 →
T(n) = O(nk)
96
Prof. Ossama Ismail
 Questions ???

97
Prof. Ossama Ismail