1

BASICS OF ANALYSIS
OF ALGORITHMS
 OUTLINES
2

 Introduction
 Factors that affect running time
 Issues in Algorithm Analysis
 Running Time computation technique
 Worst case Running Time
 Best case running time
 Average case running time
 Amortized Analysis

2
 OUTLINES
3

 Running time Analysis for

 Insertion sort Algorithms
 Which Operation should be focused to Analyze Running
Time
 Selection sort Algorithms
 bubble sort Algorithms
 Order of Growth of Algorithms (Growth rate)
 Big-Oh
 Big-Omega
 Big-Theta

3
 INTRODUCTION
4

 Analyzing an algorithm has come to mean predicting the resources

that the algorithm requires. Algorithm analysis refers to the
process of determining the amount of computing time and
storage space required by different algorithms
 Algorithm analysis mainly focus on resources such as
 computational time (most important)
 computational memory

 communication bandwidth.

 Generally, by analyzing several candidate algorithms for a problem,

the most efficient one can be identified.
 Such analysis may indicate more than one viable candidate, but
several inferior algorithms are usually discarded in the process.

4
 FACTORS THAT AFFECT RUNNING TIME
5

 There are different factors that affect the running time of an

algorithm
1. The Computer Architecture
 affects the processing speed of the system as it differs in CPU
speed, cache memory, RAM, register, instruction set, etc
2. The Compiler Used:
 different compilers of the same programming language result
major difference in the CPU requirement to solve a given
problem on a given data as they have variation on the level of
optimization of code

5
 FACTORS THAT AFFECT RUNNING TIME
6

3. Algorithm used
 different algorithm require different running time for the
same problem under the same environment

4. The input Size

 Usually as the input size increases the running time
requirement increases

6
 FACTORS THAT AFFECT RUNNING TIME
7

5. The process load of the machine

 Micro processor work in time sharing principle

 As more process are scheduled in the machine, your

program running time would be affected
 The actual number of instructions executed for your
program might not be affected by the process load but the
duration it takes is

7
 ISSUES IN ALGORITHM ANALYSIS
8

 It is not possible to generalize one Algorithm is

Inherently better than Others.
 Algorithm Analysis results should be applicable to All
Machines
 Algorithm Analysis results should apply regardless of
Input Size
 We can analyze only correct algorithms

8
 ALGORITHM RUNNING TIME
9

 Running time is the time an algorithm takes to

accomplish its task
 Running time computation might involve
computation of best case, worst case, and average
case analysis as described below

9
 WORST CASE RUNNING TIME
10

 Worst-case running time of an algorithm is the

maximum number of steps an algorithm will take on
a given data size n
 It is the longest running time or the upper bound of
the running time for any input size n
 Worst case running time guarantee that, the algorithm
will never take longer time

10
 BEST CASE RUNNING TIME
11

 Best-case running time refers to the smallest number of

steps an algorithm will take on a given size data n
 It is the shortest running time or the lower bound of the
running time for any input size n
 It is not good to use this measure for decision making
 Best case running time will help to reject an algorithm if
its best case is expensive

11
 AVERAGE CASE RUNNING TIME
12

 Falls between the best case and the worst case extremes
 It may not be simple to compute the average case for a given
algorithm
 Average case computation can be made in three ways
 Just by adding best case and worst case and divide by two
 This is simple but not appropriate
 By evaluating on a real environment with the representative data
 By statistical means by randomly generating test data

12
 AVERAGE CASE RUNNING TIME
13

 Statistical approach
 Established by considering possible inputs to an algorithm
 Determining the number of steps performed by the algorithm
for each input
 Adding the number of steps for all the inputs and dividing by
the number of inputs

13
 AMORTIZED ANALYSIS
14

 Amortized analysis is a better option to get the feel of the

average case time complexity of an algorithm
 In this approach, for a given input size n, all possible set of input
data are assumed to be equally likely.
 Hence, we will consider the actual cost of each of the sample
inputs and take their average (which is the amortized) cost.
 Amortized analysis shows the long term cost of the algorithm
 Amortized analysis may be very difficult but indicative if
feasible

14
 AMORTIZED ANALYSIS
15

 

15
RUNNING TIME COMPUTATION TECHNIQUE
16

 It is not possible to compute the actual time as it is

affected by various factors
 Simple statement might take a different amount of time
than another line,
 But we shall assume that each simple statement
execution takes the same time unit (1 unit time) of the
CPU
 Simple statement is considered as
 assignment statement
 Logical or relational or arithmetic expressions
16
RUNNING TIME COMPUTATION TECHNIQUE
17

 Calling a function will incur a cost equals to the cost of

the statements inside the function plus some constant L.
 But we might ignore again the constant L
 Recursion will have recurrence relation (will be
discussed in the next chapter)
 The following heuristic would help to analyze branching
and looping statements so as to determine the number of
simple statement

17
RUNNING TIME COMPUTATION TECHNIQUE
18

Analysis of branching statements

 Branching statement with K alternatives might be used to
determine the best and worst case
 The best case is the branch that leads into the smallest
possible running time computation among the alternative
branches
 The worst case is the branch that leads into the smallest
possible running time computation among the alternative
branches

18
RUNNING TIME COMPUTATION TECHNIQUE
19

Analysis of while loop

 In a while loop, when the condition will be true or false
determine the best case and the worst case
 A while loop condition might be simple relational
expression or logical expression involving two or more
relational expression connected by logical operators
 We will discuss different case to better understand the
situation

19
RUNNING TIME COMPUTATION TECHNIQUE
20

Case I: If condition is relational expression

 In this case, the loop iterate K times and hence condition
will be checked K + 1 times
 The value of K is determined by the control variable(s) and
its modification logic
 Example while( i < N)

20
RUNNING TIME COMPUTATION TECHNIQUE
21

Case II: If condition is logical expression where two or more

relational expressions are connected by AND
 In this case, the loop iterate if all relational expressions are evaluated to
true
 In this case, we might identify the best case and the worst case situations
 The best case is determined as the minimum number of iteration
possible due to the condition set
 This took the assumption data might fit to push the condition to fail as
early as possible
 The worst case is determined as the maximum number of iteration
possible due to the condition set
 This took the assumption data might fit to push the condition to
succeed all the time

21
RUNNING TIME COMPUTATION TECHNIQUE
22

Case II: condition is logical expression where two or more

relational expressions are connected by AND
Example the while loop of the insertion sort
ji-1
while(j >=0 and A[j] > key)
the relational expression A[j] >key might fail immediately and
hence minimum iteration is 0, or
it succeeded all the time and failed when j becomes -1 due to the
relational expression j >= 0 and hence the maximum number of
iteration becomes i.
 So best case iteration is 0 and worst case iteration is i and the
condition will be checked 1 times in the best case and i+1 time in
the worst case

22
RUNNING TIME COMPUTATION TECHNIQUE
23

Case III: condition is logical expression where two or more

relational expressions are connected by OR
 In this case, the loop iterate if at least one relational expression is
evaluated to true
 In this case, we might identify the best case and the worst case situations
 The best case is determined as the minimum number of iteration
possible due to the condition set
 This took the assumption data might fit to push all the relational
expression to fail as early as possible
 The worst case is determined as the maximum number of iteration
possible due to the condition set
 This took the assumption data might fit to push at least one of the
relational expression to succeed at a time

23
RUNNING TIME COMPUTATION TECHNIQUE
24

Case III: condition is logical expression where two or more

relational expressions are connected by OR
Example:
 Consider the while loop
j  i-1
A[0]  key + 1
B[o]  key -1
while (A[j] > key OR B[j] < key ){
:
:
j--

 The relational expressions might fail at the beginning (best case) or failed
when j become 0 (worst case)

24
RUNNING TIME COMPUTATION TECHNIQUE
25

Analysis of a for loop

 A for loop of the form
for i  s to m
conduct 1 initialization, (m-s+2) comparison and (m-s
+1) incrimination of i
 Hence the above for loop require 2(m-s + 2) simple
statement operation
 You might take just m-s operation without loss of
generality to simplify the number of operation

25
RUNNING TIME COMPUTATION TECHNIQUE
26

Analysis
  of a for loop
In general, a for loop translates to a summation
The index and bounds of the summation are the same as
the index and bounds of the for loop

for i = 1 to N do
sum = sum + i;
end for loop

26
RUNNING TIME COMPUTATION TECHNIQUE
27

 Nested for loops translate into multiple summations,

one for each for loop.
for i = 1 to N do
for j = 1 to M do
sum = sum + i + j ;
end inner for
end outer for

27
RUNNING TIME COMPUTATION TECHNIQUE
28

Add the running times of the separate blocks of your

algorithm
for i = 1 to N do
sum = sum + i;
end for
for i = 1 to N do
for j = 1 to N do
sum = sum + i + j;
end inner for
end outer for
28
 1.2.2. Algorithm Analysis Concepts

Algorithm analysis refers to the process of determining the amount of

computing time and storage space required by different algorithms.
In other words, it’s a process of predicting the resource requirement
of algorithms in a given environment.
In order to solve a problem, there are many possible algorithms. One
has to be able to choose the best algorithm for the problem at hand
using some scientific method. To classify some data structures and
algorithms as good, we need precise ways of analyzing them in terms
of resource requirement.
The main resources are:
Running Time
Memory Usage
Communication Bandwidth
Running time is usually treated as the most
important since computational time is the most
precious resource in most problem domains.
There are two approaches to measure the efficiency of
algorithms:
Empirical: Programming competing algorithms and
trying them on different instances.
Theoretical: Determining the quantity of resources
required mathematically (Execution time, memory
space, etc.) needed by each algorithm.
However, it is difficult to use actual clock-time as a
consistent measure of an algorithm’s efficiency,
because clock-time can vary based on many things.
For example,
Specific processor speed
Current processor load
Specific data for a particular run of the program
 Input Size
 Input Properties

Operating Environment
 1.2.3. Complexity Analysis

Complexity Analysis is the systematic study of the cost of

computation, measured either in time units or in operations
performed, or in the amount of storage space required.
The goal is to have a meaningful measure that permits
comparison of algorithms independent of operating
platform.
There are two things to consider:
Time Complexity: Determine the approximate number of
operations required to solve a problem of size n.
Space Complexity: Determine the approximate memory
required to solve a problem of size n.
Complexity analysis involves two distinct phases:
Algorithm Analysis: Analysis of the algorithm or data
structure to produce a function T (n) that describes the
algorithm in terms of the operations performed in order to
measure the complexity of the algorithm.
Order of Magnitude Analysis: Analysis of the function T
(n) to determine the general complexity category to which it
belongs.

There is no generally accepted set of rules for algorithm

analysis. However, an exact count of operations is commonly
used.
 1.2.3.1. Analysis Rules:

1. We assume an arbitrary time unit.

2. Execution of one of the following operations takes time 1:
Assignment Operation
Single Input/Output Operation
Single Boolean Operations
Single Arithmetic Operations
Function Return
3. Running time of a selection statement (if, switch) is the time for the condition evaluation +
the maximum of the running times for the individual clauses in the selection.
4.Loops: Running time for a loop is equal to the running time for the statements inside the loop
* number of iterations.
The total running time of a statement inside a group of nested loops is the running time of the
statements multiplied by the product of the sizes of all the loops.
For nested loops, analyze inside out.
Always assume that the loop executes the maximum number of iterations possible.
5.Running time of a function call is 1 for setup + the time for any parameter calculations + the
time required for the execution of the function body.
Examples:
int count()
{
int k=0;
cout<< “Enter an integer”;
cin>>n;
for (i=0;i<n;i++)
k=k+1;
return 0;
}
Time Units to Compute
-------------------------------------------------
1 for the assignment statement: int k=0
1 for the output statement.
1 for the input statement.
In the for loop:
 1 assignment, n+1 tests, and n increments.
 n loops of 2 units for an assignment, and an addition.
 1 for the return statement.
-------------------------------------------------------------------
T (n)= 1+1+1+(1+n+1+n)+2n+1 = 4n+6 = O(n)
2. int total(int n)
{
int sum=0;
for (int i=1;i<=n;i++)
sum=sum+1;
return sum;
}
Time Units to Compute
-------------------------------------------------
1 for the assignment statement: int sum=0
In the for loop:
 1 assignment, n+1 tests, and n increments.
 n loops of 2 units for an assignment, and an addition.
 1 for the return statement.
-------------------------------------------------------------------
T (n)= 1+ (1+n+1+n)+2n+1 = 4n+4 = O(n)
void func()
{
int x=0;
int i=0;
int j=1;
cout<< “Enter an Integer value”;
cin>>n;
while (i<n){
x++;
i++;
}
while (j<n)
{
j++;
}
}
Time Units to Compute
-------------------------------------------------
1 for the first assignment statement: x=0;
1 for the second assignment statement: i=0;
1 for the third assignment statement: j=1;
1 for the output statement.
1 for the input statement.
In the first while loop:
 n+1 tests
 n loops of 2 units for the two increment (addition) operations
In the second while loop:
 n tests
 n-1 increments
-------------------------------------------------------------------
T (n)= 1+1+1+1+1+n+1+2n+n+n-1 = 5n+5 = O(n)
4. int sum (int n)
{
int partial_sum = 0;
for (int i = 1; i <= n; i++)
partial_sum = partial_sum +(i * i * i);
return partial_sum;
}
Time Units to Compute
-------------------------------------------------
 1 for the assignment.
 1 assignment, n+1 tests, and n increments.
 n loops of 4 units for an assignment, an addition, and two multiplications.
 1 for the return statement.
-------------------------------------------------------------------
T (n)= 1+(1+n+1+n)+4n+1 = 6n+4 = O(n)