Introduction

1. Algorithms

2. Order

3. Analysis of Algorithm

4. Some Mathematical Background

Topic: Introduction 1
What is an algorithm?

An algorithm can be defined as a well-defined computational

procedure that takes some values, or the set of values, as an
input and produces some value, or the set of values, as an
output.
An algorithm is thus a sequence of computational
steps that transform the input into output. It describes specific
computational procedures for achieving the input-output
relationship.

For Example: We need to sort the sequence of number into

ascending order. Here is how we define the sorting problem.

Topic: Introduction 2
Thus an algorithm is
A simple, unambiguous, mechanical
procedure to carry out some task.

Why algorithm instead of program?

1. Writing an algorithm is simpler (we don’t
need to worry about the detailed
implementation, or the language syntax).

2. An algorithm is easier to read than a

program written in, for instance, C.
Topic: Introduction 3
 Need of Algorithm:
• 1. To understand the basic idea of the problem.
• 2. To find an approach to solve the problem.
• 3. To improve the efficiency of existing techniques.
• 4. To understand the basic principles of designing the
algorithms.
• 5. To compare the performance of the algorithm with
respect to other techniques.
• 6. It is the best method of description without describing
the implementation detail.

Topic: Introduction
4
 Need of Algorithm:
• 7. The Algorithm gives a clear description of requirements and goal of
the problem to the designer.
• 8. A good design can produce a good solution.
• 9. To understand the flow of the problem.
• 10. To measure the behavior (or performance) of the methods in all
cases (best cases, worst cases, average cases)
• 11. With the help of an algorithm, we can also identify the resources
(memory, input-output) cycles required by the algorithm.
• 12. With the help of algorithm, we convert art into a science.
• 13. To understand the principle of designing.
• 14. We can measure and analyze the complexity (time and space) of
the problems concerning input size without implementing and running
it; it will reduce the cost of design.

Topic: Introduction 5
 Algorithm vs Program:
A finite set of instructions that specifies a sequence of operations to be
carried out to solve a specific problem of a class of problem is called
an algorithm.
On the other hand, the Program doesn't have to satisfy the finiteness
condition.
For example, we can think of an operating system that continues in a
"wait" loop until more jobs are entered. Such a program doesn't
terminate unless the system crashes.

Given a Problem to solve, the design Phase produces an algorithm, and

the implementation phase then generates a program that expresses the
designed algorithm. So, the concrete expression of an algorithm in a
particular programming language is called a program.
Topic: Introduction 6
How to represent an algorithm?

1. Give a description in your own language,

e.g. English, Spanish, …

2. Pseudo code

3. Graphical

Topic: Introduction 7
Example – multiplying two positive integers A and
B

For example: 45*19

Usually:
45
19 (x
405
45 (+
855
Topic: Introduction 8
A different algorithm:

Multiplier Multiplicand Result

(A/2) (B*2) (pick numbers in column
2 when the corresponding
number under the
multiplier is odd)

45 19 19
22 38
11 76 76
5 152 152
2 304
1 608 608 (+
855

Topic: Introduction 9
An instance of a problem is a specific assignment of values to
the parameters.

This algorithm can be used for multiplying any two positive

integers, we say that (45, 19) is an instance of this problem.
Most problems have infinite collection of instances.

It’s ok to define the domain (i.e. the set of instances) to be

considered, and the algorithm should work for all instances in
that domain.

Although the above algorithm will not work if the first operand
is negative, this does not invalidate the algorithm since (-45, 19)
is not an instance of the problem being considered.

Topic: Introduction 10
 Order:
Usually we use the frequency count to compare algorithms.
Consider the following 3 programs:

(a) (b) (c)

x←x+y for i ← 1 to n do for i ← 1 to n do
x←x+y for j ← 1 to n do
end x←x+y
end
end

The frequency count of stmt x ← x + y is 1, n, n2.

No matter which machine we use to run these programs, we know
that the execution time of (b) is n times the execution time of (a).
Topic: Introduction 11
 Algorithm Analysis

•How long an algorithm

Analysis Helps
will take for a given
problem set

•How much memory

required

Topic: Introduction 12
 Examples

Sorting Algorithm Operations performed Input N Values

Comparisons

Matrix Operations performed Input N * N Values

Multiplications Arithmatic

There are number of algorithm to solve above problem

Analysis of Algorithm gives tools to choose between the algorithm

Topic: Introduction 13
 Algorithm Design Techniques:

The following is a list of several popular design approaches:

1. Divide and Conquer Approach
2. Greedy Technique
3. Dynamic Programming
4. Branch and Bound
5. Randomized Algorithms
6. Backtracking Algorithm

Topic: Introduction 14
1. Divide and Conquer Approach:
It is a top-down approach. The algorithms which follow the divide
& conquer techniques involve three steps: Divide the original
problem into a set of sub problems. Solve every sub problem
individually, recursively. Combine the solution of the sub problems
(top level) into a solution of the whole original problem.

Topic: Introduction 15
2. Greedy Technique:
Greedy method is used to solve the optimization problem. An optimization
problem is one in which we are given a set of input values, which are required
either to be maximized or minimized (known as objective), i.e. some constraints
or conditions. Greedy Algorithm always makes the choice (greedy criteria)
looks best at the moment, to optimize a given objective.
The greedy algorithm doesn't always guarantee the optimal solution however it
generally produces a solution that is very close in value to the optimal.

Topic: Introduction 16
3. Dynamic Programming:
Dynamic Programming is a bottom-up approach we solve all
possible small problems and then combine them to obtain solutions
for bigger problems. This is particularly helpful when the number
of copying sub problems is exponentially large. Dynamic
Programming is frequently related to Optimization Problems.

Topic: Introduction 17
4. Branch and Bound:
Branch and bound is an algorithm design paradigm which is
generally used for solving combinatorial optimization problems.
These problems are typically exponential in terms of time
complexity and may require exploring all possible permutations in
worst case. The Branch and Bound Algorithm technique solves
these problems relatively quickly.
In Branch & Bound algorithm a given sub problem, which cannot
be bounded, has to be divided into at least two new restricted sub
problems. Branch and Bound algorithm are methods for global
optimization in non-convex problems. Branch and Bound
algorithms can be slow, however in the worst case they require
effort that grows exponentially with problem size, but in some
cases we are lucky, and the method coverage with much less effort.

Topic: Introduction 18
Topic: Introduction 19
5. Randomized Algorithms:
A randomized algorithm is defined as an algorithm that is allowed
to access a source of independent, unbiased random bits, and it is
then allowed to use these random bits to influence its computation.

Topic: Introduction 20
6. Backtracking Algorithm:
Backtracking Algorithm tries each possibility until they find the
right one. It is a depth-first search of the set of possible solution.
During the search, if an alternative doesn't work, then backtrack to
the choice point, the place which presented different alternatives,
and tries the next alternative.

Topic: Introduction 21
Loop Invariants

•This is a justification technique, justy – “Algorithm is correct”

•We use loop invariant that helps us to understand why an

algorithm is correct.

• with the help of Loop Invariants we can prove correctness of

algorithm with loops

•Loop Invariant is the propeerty which will remain True

•Before Itteration
•At each and Loop
•After

Topic: Introduction 22
Loop invariants
Loop Invariants has three properties:-

Initialization: It is true prior to the Initialization of the first

iteration.

Maintenance: If it is true before an iteration of loop , it remains

true before the next iteration of loop.

Termination: When the loop finishes, the loop invariants provides

a useful property that helps us demonstrate that the algorithm is
correct.

Topic: Introduction 23
To prove statement S about a loop is correct, define S concerning
series of smaller statement S0 ,S1... .Sk where,

(i) The initial claim so is true before the loop begins.

(ii) If Si-1 is true before iteration i begin, then one can show that
Si will be true after iteration i is over.
(iii) The final statement Sk implies the statement S that we wish to
justify as being true.
Insetion Sort
1.For j=2 to A Length
2. key=A[j]
3. // Insert A[j] in the sorted sequence A[1…,j-1]
4. i=j-1
5. While i>0 && A[i]> key
6. A[j+1] = A[i]
7. i=i-1
8. A[i+1]=key
Topic: Introduction 24
Initialization: It is true prior to the Initialization of the first
iteration.
Insetion Sort
1.For to A Length
j=2 ie: prior to the first iteration. 2. key=A[j]
3. // Insert A[j] in the sorted
Therefore at time A[1…..j-1]= A[1] sequence A[1…,j-1]
2. i=j-1
Array of single element is always sorted 3. While i>0 && A[i]> key
4. A[j+1] = A[i]
5. i=i-1
6. A[i+1]=key

Topic: Introduction 25
Maintainence:
( Mathematical Induction)
Insetion Sort
*Assume that it is true before a particular iteration. 1.For j=2 to A Length
2. key=A[j]
:Then prove that it remains true before beginning of
3. // Insert A[j] in the sorted
Next iteration.
sequence A[1…,j-1]
2. i=j-1
If suppose at jth iteration

Assumption: at (j-1)th iteration it is true.

While loop- moves all elements to find 6. A[i+1]=key
correct position of Aj
When we are placing Aj at the correct position
Array A[1………., Aj-1] is sorted
At the beginning of next iteration j will j+1 and so on
so..,Array A[1………., Aj] is sorted
Which means loop invariant hold true
Topic: Introduction 26
Termination:
Loops For loop terminate when Insetion Sort
J > n (ie: j=n+1)
2. key=A[j]
The sub array is A[1……n] by definition is in sorted 3. // Insert A[j] in the sorted
Order sequence A[1…,j-1]
2. i=j-1
Proves Array is sorted 3. While i>0 && A[i]> key
4. A[j+1] = A[i]
5. i=i-1
Proves that Insertion Sort is correct 6. A[i+1]=key

Which means loop invariant hold true

Topic: Introduction 27