ADVANCED ANALYSIS OF ALGORITHMS

Dr. Javed Iqbal Bangash

Types of algorithm

 Iterative algorithm
 Recursive algorithm
How to Write Iterative Algorithm? Example-1
 Algorithm 1.1: Sequential Search Pseudocode
 Problem: Is the key x in the array S of n keys?
 Inputs: positive integer n, array of keys S indexed from 1 to n, and a key
x.
 Outputs: location, the location of x in S (0 if x is not in S.)
 void seqsearch (int n, const keytype S [ ], keytype x, index & location)
{ location = 1;
while (location <= n and S[location] != x)
location ++;
if (location > n)
location=0; }
How to Write Iterative Algorithm? Example-2
 Algorithm 1.2: Add Array Members
 Problem: Add all the numbers in the array S of n numbers.
 Inputs: positive integer n, array of numbers S indexed from 1 to n.
 Outputs: sum, the sum of the numbers in S.
number sum (int n, const number S [ ])
{ index i;
number result;
result = 0;
for (i = 1; i <= n; i++)
result = result + S[i];
return result;
}
How to Write Iterative Algorithm? Example-3
 Algorithm 1.3: Exchange Sort
 Problem: Sort n keys in non-decreasing order.
 Inputs: positive integer n, array of keys S indexed from 1 to n..
 Outputs: the array S containing the keys in non-decreasing order.
void exchangesort (int n, keytype S[])
{
index i, j ;
for (i =1; i <= 1; i++)
for (j = i+l; j <= n; j++)
if (S[j] < S[i])
exchange S[i] and S[j];
}
How to Write Iterative Algorithm? Example-4
 Algorithm 1.4: Matrix Multiplication
 Problem: Determine the product of two n x n matrices.
 Inputs: A positive integer n, two-dimensional arrays of numbers A and

B, each of which has both its rows and columns indexed from I to n.
 Outputs: A two-dimensional array of numbers C, which has both its

rows and columns indexed from 1 to n, containing the product of A and

B.
void matrixmult (int n, number A[][], number B[][], number C[][])
{
index i, j, k; ;
for (i =1; i <= 1; i++)
for (j = i+l; j <= n; j++)
if (S[j] < S[i])
exchange S[i] and S[j];
How to Write Iterative Algorithm? Example-4 (Conti…)

void matrixmult (int n, number A[][], number B[][], number C[][])

{
index i, j, k; ;
for (i=1; j<= n; j++)
for (j=1; j<= n; j++)
{
C[ i ] [ j ] = 0;
for (k=1; k<= n; k++)
C[ i ] [ j ] = C[ i ] [ j ] + A [ i ] [ k ] * B [ k ] [ j ];
}
}
The Importance of Developing Efficient Algorithms

 Let's compare the work done by Sequential Search and

Binary Search.
 For focus we will determine the number of comparisons
done by each algorithm.
 If the array S contains 32 items and x is not in the array,
Sequential Search compares x with all 32 items before
determining that x is not in the array.
 In general, it does n comparisons to determine that x is
not in an array of size n.
The Importance of Developing Efficient Algorithms (Conti…)

 We will assume the algorithm is implemented in such a

manner that there is only one comparison.
 Figure 1.1 shows that the algorithm does 6 comparisons
when x is larger than all the items in an array of size 32.

 Notice that 6 = log2 (32 + 1).

Table 1.1: The number of comparisons done by Sequential Search
and Binary Search when x is larger than all the array items

Number of Comparisons by Number of Comparisons

Array Size Sequential Search by Binary Search
128 128 8

1,024 1,024 11

1,048,576 1,048,576 21

4, 294, 967, 296 4, 294, 967, 296 33

Fibonacci Sequence

 Fibonacci Sequence is defined as:

f0  0
f1  1
f n  f n 1  f n  2 n  2
 Computing the first few terms
f 2  f1  f 0  1  0  1
f 3  f 2  f1  1  1  2
f 4  f3  f 2  2  1  3
f 5  f 4  f 3  3  2  5, etc.
Fibonacci Sequence (Conti…)
 Algorithm 1.6: nth Fibonacci Term (Recursive)
 Problem: Determine the nth term in the Fibonacci sequence.
 Inputs: A nonnegative integer n.
 Outputs: fib, the nth term of the Fibonacci sequence.
 int fib (int n)
{
if (n <= 1)
return n;
else
return fib (n - 1) + fib (n-2);
}
Fibonacci Sequence (Conti…)
 Algorithm 1.7: nth Fibonacci Term (Iterative)
 Problem: Determine the nth term in the Fibonacci sequence.
 Inputs: A nonnegative integer n.
 Outputs: fib, the nth term of the Fibonacci sequence.
 int fib2 (int n)
{ index i; int f[0 .. n]; f[ 0 ] = 0;
if (n > 0) {
f[ 1 ] = 1;
for (i = 2; i<= n; i++)
f[ i ] = f[i - 1] + f [i -2 ]; }
return f[ n ]; }
 Fibonacci Sequence (Conti…)

Table 1.2: A comparison of Algorithms 1.6 and 1.7

n n+1 2n/2 Execution Time Using Lower Bound on Execution Time

Algorithm 1.7 Using Algorithm 1.6
40 41 1,048,576 41 ns[*] 1048 μs[†]
60 61 1.l × 1O9 61 ns 1s

80 81 1.1 × 1012 81 ns 18 min

100 101 1.1 × 1015 101 ns 13 days

120 121 1.2 × 1018 121 ns 36 years

160 161 1.2 × 1024 161 ns 3.8 × 107 years

200 201 1.3 × 1030 201 ns 4 × 1013 years

[*]
1 ns = 10-9 second.
[†]
1 μs = 10−6 second.
Analysis of Algorithms

 We analyze algorithms with the intention of improving

them, if possible, and for choosing among several
available for a problem.
 We will use the following criteria:
 Correctness
 Complexity
 Space usage
 Simplicity
 Optimality
Correctness

 An algorithm is said to be correct if, for every input

instance, it halts with the correct output.
 Can incorrect algorithms be sometimes useful:
 Three major steps are involved in establishing the
correctness of an algorithm:
 What “correct” means?
 A precise statement about the characteristics of the inputs
(preconditions), and what result it is to produce for each input
(postconditions).
Correctness (Conti…)

 Steps (Conti…):
 If the preconditions are satisfied, the postconditions will be
true when the algorithm terminates.
 Loop invariants are used to help us understand why an
algorithm is correct.
Complexity – Amount of work done

 A measure of work that tells something about the

efficiency of an algorithm independent of:
 Computer
 Programming Language
 Programmer
 Implementation Details
 A simple algorithm consists of:
 Initialization Instructions
 A Loop
 Fundamental (Basic) Operation
Complexity – Amount of work done (Conti…)

 Time Complexity Analysis:

 is the determination of how many times the basic operation is
done for each value of the input size.
 Time Complexity Analysis Techniques: will be discussed
in detail later.
 Best-Case Time Complexity Analysis
 Worst-Case Time Complexity Analysis
 Average-Case Time Complexity Analysis
 Every-Case Time Complexity Analysis
Space Usage

 A program requires storage space for the instructions, the

constants and variables, and the input data. It may also
use some extra space for manipulating the data and
storing information needed to carry out its computations.
 If the amount of extra space is constant, the algorithm is
said to work in place.
 If the input can be represented in various forms, then we
will consider the space required for the input itself as
well as any extra space used.
Simplicity

 It is not always the case that the simplest way to solve a

problem is not the most efficient. Yet simplicity is a
desirable feature.
 It makes:
 writing
 debugging
 & modifying
 of a program easier.
 If the program is to be used very often, its efficiency will
probably be the determining factor.
Optimality

 Each problem has inherent complexity, that is, there is

some minimum amount of work required to solve it.
 Therefore, an algorithm can’t be improved beyond a
certain point.
 An algorithm is optimal (in the worst case) if there is no
algorithm in the class under study that performs fewer
basic operations (in the worst case).
Optimality (Conti…)

 A “tight” lower bound, one that matches the upper bound

for an algorithm, assures that further improvements
cannot be found.
 An algorithm is optimal, if W(n) = A(n) = B(n).
 For example the binary search algorithm is optimal in
the class of algorithms that perform comparisons only.