CSE-243: Algorithm Design and Analysis

 Lecture 03: Complexity Analysis

Instructor Teaching Assistants
Md. Sabir Hossain Rakibul Islam
Lecturer Id: 1604060
Dept. of CSE, CUET Email: u1604060@student.cuet.ac.bd
Mobile: 01737143868/01882826575 Saiful Islam Rimon
Email: sabirndc08cuet10@gmail.com
Id: 1604061
Email : u1604061@student.cuet.ac.bd
Acknowledgment: Most of the content of this slide is adopted from the book: Fundamentals of Computer Algorithms, Second
Edition, Ellis Horowitz, Sanguthevar Rajasekaran, Sartaj Sahni, ISBN 10: 8173716129 / ISBN 13: 9788173716126, Published by
Universities Press/Orient BlackSwan, 2008.
“Do not cry over spilt milk”
Today’s Target

 Complexity Analysis
 Space Complexity
 Time Complexity
 Related Example for Counting Time Complexity
 Assignment
Algorithmic Performance
There are two aspects of algorithmic performance:
 Time
 Instructions take time.
 How fast does the algorithm perform?
 What affects its runtime?
 Space
 Data structures take space
 What kind of data structures can be used?
 How does choice of data structure affect the runtime?
 We will focus on time:
 How to estimate the time required for an algorithm
 How to reduce the time required
Md. Sabir Hossain, CSE, CUET 4
 Analysis of Algorithms
 Analysis of Algorithms is the area of computer science
that provides tools to analyze the efficiency of different
methods of solutions.
 How do we compare the time efficiency of two
algorithms that solve the same problem?
Naïve Approach: implement these algorithms in a
programming language (e.g. C/C++/Java), and run them
to compare their time requirements. Comparing the
programs (instead of algorithms) has difficulties.

Md. Sabir Hossain, CSE, CUET 5

03 Factors in consideration
 How are the algorithms coded?
 Comparing running times means comparing the
implementations.
 We should not compare implementations, because they are
sensitive to programming style that may cloud the issue of
which algorithm is inherently more efficient.
 What computer should we use?
 We should compare the efficiency of the algorithms
independently of a particular computer.
 What data should the program use?
 Any analysis must be independent of specific data.
 Analysis of Algorithms
 When we analyze algorithms, we should
employ mathematical techniques that
analyze algorithms independently of specific
implementations, computers, or data.

 To analyze algorithms:
 First, we start to count the number of significant
operations in a particular solution to assess its
efficiency.
 Then, we will express the efficiency of
algorithms using growth functions.
Md. Sabir Hossain, CSE, CUET 7
Running Time, tp (Time Complexity)
Running time of an algorithm for a given input is
• The number of steps executed by the algorithm on that input.
• Often referred to as the complexity of the algorithm.

tp(n) can be expressed as:

Where,
• n = instance characteristics
• Ca = time needed for Addition
• Cs = time needed for Subtraction
• Cm = time needed for Multiplication
• Cd = time needed for Division
• ADD,SUB,MUL,DIV are functions whose values are the numbers of addition,
subtractions, multiplications, divisions and so on.
RAM Model
 Generic single-processor model.
 Supports simple constant-time instructions found in
real computers.
 Arithmetic (+, –, *, /, %, floor, ceiling).
 Data Movement (load, store, copy).
 Control (branch, subroutine call).
 Run time (cost) is uniform (1 time unit) for all
simple instructions.
 Memory is unlimited.
 Flat memory model – no hierarchy.
 Access to a word of memory takes 1 time unit.
 Sequential execution – no concurrent operations.
General Rules for Estimation
 Consecutive Statements: Just add the running times of
those consecutive statements.
 If/Else: Never more than the running time of the test
plus the larger of running times of S1 and S2.
 Loops: The running time of a loop is at most the running
time of the statements inside of that loop times the
number of iterations.
 Nested Loops: Running time of a nested loop containing
a statement in the inner most loop is the running time of
statement multiplied by the product of the sized of all
loops.
Md. Sabir Hossain, CSE, CUET 10
 The Execution Time of Algorithms
 Each operation in an algorithm (or a program) has a cost.
 Each operation takes a certain of time.

count = count + 1;  take a certain amount of time, but it is constant

A sequence of operations:

count = count + 1; Cost: c1

sum = sum + count; Cost: c2

 Total Cost = c1 + c2

Md. Sabir Hossain, CSE, CUET 11

 The Execution Time of Algorithms (cont.)

Example: Simple If-Statement

Cost Times
if (n < 0) c1 1
absval = -n c2 1
else
absval = n; c3 1

Total Cost <= c1 + max(c2,c3)

Md. Sabir Hossain, CSE, CUET 12

 The Execution Time of Algorithms (cont.)
Example: Simple Loop
Cost Times
i = 1; c1 1
sum = 0; c2 1
while (i <= n) { c3 n+1
i = i + 1; c4 n
sum = sum + i; c5 n
}

Total Cost = c1 + c2 + (n+1)*c3 + n*c4 + n*c5

 The time required for this algorithm is proportional to n
Md. Sabir Hossain, CSE, CUET 13
 The Execution Time of Algorithms (cont.)
Example: Nested Loop
Cost Times
i=1; c1 1
sum = 0; c2 1
while (i <= n) { c3 n+1
j=1; c4 n
while (j <= n) { c5 n*(n+1)
sum = sum + i; c6 n*n
j = j + 1; c7 n*n
}
i = i +1; c8 n
}

Md. Sabir Hossain, CSE, CUET 14

 The Execution Time of Algorithms (cont.)
Example: Nested Loop
Cost Times
i=1; c1 1
sum = 0; c2 1
while (i <= n) { c3 n+1
j=1; c4 n
while (j <= n) { c5 n*(n+1)
sum = sum + i; c6 n*n
j = j + 1; c7 n*n
}
i = i +1; c8 n
}
Total Cost = c1 + c2 + (n+1)*c3 + n*c4 + n*(n+1)*c5+n*n*c6+n*n*c7+n*c8

Md. Sabir Hossain, CSE, CUET 15

 The Execution Time of Algorithms (cont.)
Example: Nested Loop
Cost Times
i=1; c1 1
sum = 0; c2 1
while (i <= n) { c3 n+1
j=1; c4 n
while (j <= n) { c5 n*(n+1)
sum = sum + i; c6 n*n
j = j + 1; c7 n*n
}
i = i +1; c8 n
}
Total Cost = c1 + c2 + (n+1)*c3 + n*c4 + n*(n+1)*c5+n*n*c6+n*n*c7+n*c8
 The time required for this algorithm is proportional to n2

Md. Sabir Hossain, CSE, CUET 16

Counting Total Steps
An algorithm to count the total number
of words in a sentence
1. START
2. READ string s
3. i=0,count=0
4. WHILE i<= size of string s ? DO
5. IF s[i]==' ' ? THEN
6. count++
7. ENDIF
8. i++
9. ENDWHILE
10. WRITE Count+1
11. END
An algorithm to find transpose of a
matrix
1. ENTER
2. READ row,column
3. READ the elements of two dimentional array
Arr[row][column]
4. i=0, j=0
5. REPEAT
6. REPEAT
7. WRITE Arr[i][j]
8. j++
9. TO j==row
10. i++
11. TO i==column
12. RETURN
An algorithm to sort even and odd
elements of array separately
1. ENTER
2. READ element Arr[]
3. n= sizeof Arr
4. l=0,k=0,r=n-1
5. WHILE l<r DO
6. l++,k++
7. ENDWHILE
8. WHILE Arr[r]%2==0 && l<r DO
9. r--
10. IF l<r THEN
11. swap(Arr[l],Arr[r])
12. ENDIF
13. ENDWHILE
14. sort(Arr,Arr+k)
15. sort(Arr+k+,Arr+n)
16. Arr[n]
17. RETURN
Another Way: Counting Steps

 Iterative Function for SUM

Algorithm Sum(a,n)
{
s:=0.0;
for i:= 1 to n do
s:= s+a[i]
rerurn s;
}
Counting Steps

 We can Determine number of steps needed by a

program to solve particular problem in following
way:
 Introduce a new variable “count”(global variable)
Initialize
with 0
Statements to increment “count” by appropriate
amount are introduced into the program
Counting Steps
 Iterative Function for SUM
Algorithm Sum(a,n)
{
s:=0.0;
count := count+1; // count is global; initially zero
for i:= 1 to n do
{
count:=count+1;// For for
s:= s+a[i]; count:=count+1; //For assignment
}
count=count+1; //For last time of for
count=count+1; //For the return
return s;
}
Counting Steps

 Simplified Version:
Algorithm Sum(a,n)
{
for i:= 1 to n do count := count+2;
count := count+3;
}
Written Assignment # 01

 Page No: 49 Problem No: 03-07

 Submission Deadline: 14/09/2017, 11:59 PM.
 Book: In Reference
Reference

 Fundamentals of Computer Algorithms, Second Edition, Ellis Horowitz,

Sanguthevar Rajasekaran, Sartaj Sahni, ISBN 10: 8173716129 / ISBN 13:
9788173716126, Published by Universities Press/Orient BlackSwan, 2008.
 Allah Hafez