Introduction

Prof. Dr. Md. Rafiqul Islam

Chapter 1
Basic goals for an algorithm
always correct
always terminates
very good performance

Slides# 3
What will we do?
Design and Analysis of Algorithms

Design: Develop or write algorithm which gives correct
output and minimize the cost.

Analysis: show or predict the cost of an algorithm in
terms of resources and performance.

Definition of Algorithm
It is a sequence of steps (instructions) that can
be followed to solve a problem (to perform a
task).
An algorithm takes some values as input and
produces some values as output.
An algorithm is thus a sequence of
computational steps that transform the input
into the output.

General Section of Algorithm
Usually each and every algorithm has three
sections.
Input section
where we show which data elements to be given.
Operational or processing section
This is main section and here we have to do all necessary
operations, such as computation, taking decision, calling
other procedure (algorithm) etc.
Output section
where we display the result found from the second
section.
To write an algorithm we do not strictly follow the grammar of
any particular programming language. However its language may
be near to a programming language.


Study areas of algorithm


The study of algorithm includes the following areas:
Devise algorithm
There are several techniques to devise algorithms, such as
divide and conquer, dynamic programming, greedy and back
tracking methods and so on.
By the acquiring knowledge about the above mentioned
methods we will be able to devise algorithm.
Validation of algorithm
After devising the algorithm it is necessary to show that it gives
the correct output for all possible correct input. This process is
referred as algorithm validation.
If the algorithm is simple one, the validation can be verified
using paper pencil.
Otherwise a program should be written to check the validation.

Study area [contunued]
Analysis of algorithm
When an algorithm is executed using a program it
uses the Central Processing Unit (CPU) to perform
necessary operations and memory to hold the
program and data.
Analysis of algorithm or performance analysis refers
to the task determining how much computational
time and storage (memory) are required for an
algorithm.
We can compare performance of several algorithms
for the same problem by analysis of the algorithms.
This is a challenging area and required mathematical
skill.

Complexity of algorithm
Time complexity
This complexity is related to execution
(computational) time of the algorithm.
It depends on the number of element (item)
comparisons, number of element movements
(movement of data from one place to another) and
some other computational operations such as
addition, subtraction, multiplication, division etc.
However, the complexity of an algorithm is generally
dominated by a single operation such as, the number
of element comparisons, the number of
multiplications, the number of additions and
subtractions etc.

Complexity [continued]
Space complexity
This complexity is related to space (memory) needs
for the data (data structure) used in the algorithm.
If there n data items used in an algorithm, the space
complexity of the algorithm will be proportional to
n.

Cases for analysis of algorithm
For analysis of algorithm three cases may be
considered. They are:

Best case

Average case

Worst case
Cases of algorithm analysis
Best case
In this case for a given input the algorithm executes
minimum of steps to produce the desired output.
Execution time is minimum in best case.
As for example, in a sorting problem if the data
was already stored in the target order (ascending or
descending order), then this is best case for the
sorting problem.

Cases of algorithm analysis
Average case
Here for a given input the algorithm uses the
average number of steps to produce the desired
output.
Execution time is in between minimum and
maximum.
For example in case of sorting problem the random
input may be considered as average case.

Cases of algorithm analysis
Worst case
In this case for a given input the algorithm uses the
maximum number of steps to produce the desired
output.
The worst case running time is longest(maximum)
running time for any input.
The worst case running time of an algorithm is an
upper bound on the running time for any input.

Best, Average and Worst Cases
For inputs of all sizes:
1n
2n
3n
4n
5n
6n
Input instance size
R
u
n
n
i
n
g

t
i
m
e

1 2 3 4 5 6 7 8 9 10 11 12 ..
best-case
average-case
worst-case
Example of of an Algorithm
Insertion Sort

In this method we take the data of second position first
and compare it with the data of first position.
If the data in the second position is smaller than the
data in the first position, then we shift the data in first
position to the right (to the second position) and insert
the data of second position in the first position.
Otherwise the data will remain in their own positions.
After that we take data in the third position and
compare it with the data in the second position.

Insertion Sort
If it is less than the data in the second position, then
we compare it with the data in the first position.
If it is also less than the data in the first position, then
we shift the two data to their right and place the data
of the third position in the first position.
In case if the data of the third position is less than the
data of the second position and greater than the data
of the first position, we shift the data of the second
position to its right and the third data of the third
position is inserted in the second position.
By repeating the process for the data in the forth
position, fifth position and so on, we can sort the whole
list.
Insertion Sort
Using steps we can describe the process as
follows.
1. Given a list of elements (data).
2. We have to insert a data into its correct position by
moving all data (before it) to the right (that are
greater than the data which is being considered at
this moment).
3. By repeating Step-2 for all considerable data we
can arrange the whole list in ascending order.


Pictorial view of insertion sort


Shift 17 to right.
18
17 12 18 5 7 10 8
5 17 18 12 7 10 8
5 7 8 10 12 17
5 17 18 12 7 10 8
Is 12 < 17? Yes.
Is 18 < {17, 12} ? No.
Is 5 < {18, 17, 12} ? Yes.
18 12 17 5 7 10 8
Shift {12, 17, 18} to right.
17 7 12 5 18 10 8
Is 7 < {18, 17, 12, 5} ?
7 < {18, 17, 12} and 7> 5.
Shift {12, 17, 18} to right.
Insert 7 after 5 (at blank space).
Sorted List.
19
Example of insertion sort
8 2 4 9 3 6
20
Example of insertion sort
8 2 4 9 3 6
21
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
22
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
23
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
24
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
25
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
26
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
27
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
2 3 4 8 9 6
28
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
2 3 4 8 9 6
29
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
2 3 4 8 9 6
2 3 4 6 8 9 done
Algorithm for insertion sorting


Algorithm 9.4: Algorithm for insertion sorting
1. Input Array A[1n]
2. (i). for (j = 2 to n)
{
key-value = A [j];
i = (j-1);
(ii). while (i > 0 and A[i] > key-value)
{
A[i + 1] A[i];
i = i -1;
} // end of while
A[i + 1] key-value;
} //end of for
3. Output: sorted list.

Insertion sort
Time complexity

No. of element comparisons = 1 + 2 + 3 + . + (n-1)
n n
n n
2
1
2
1
2
) 1 (
2

Therefore, complexity = O(n
2
). This is upper bound of the complexity.

The space complexity of this sorting problem is O (n). That means, the space for the date
used in the algorithm is proportional to n. Such as, the required space is n bytes or 2n or
4n bytes etc.



References
1. Introduction to algorithms, Third edition
Thomas H. Corman, Charles E. Leiserson, Ronald L.
Rivest.

2. Fundamentals of computer algorithm
Ellis Horowitz, Sartaj Sahni and Rajasekaran.

3. Data Structure Fundamentals, 2
nd
Edition
Md. Rafiqul Islam and M. A. Mottalib.

This is the end of chapter 1
THANK YOU.