Why to study Algorithms?

Interval scheduling: Pictorial representation

- It is Important for all branches of Engineering.
- Driver of technological innovation.
- Challenging (Good for the brain).
◮ Network connectivity problem [stay tuned]

(a) Input (b) Output

◮ You have a resource− A lecture room and the following constraints:
Go to Pictorial representation

0.1 Lecture j start at sj and finishes at fj .

0.2 Two lectures are compatible if they don’t overlap.
Goal: Find the maximum subset of mutually compatible lectures.
3/25 4/25

Properties/Characteristics of Algorithm Notion of an Algorithm

Definition 1 (Algorithm)
An algorithm is a sequence of unambiguous instructions for solving
An algorithm has five characteristics [1]: a problem, i.e., for obtaining a required output for any legitimate
1. Input: An algorithm has zero or more inputs: quantities that are given input in a finite amount of time [2].
to it initially before the algorithm begins, or dynamically as the algorithm
runs.
2. Output: An algorithm has one or more output.
3. Finiteness: Total number of steps used in algorithm should be finite. In
other words, the algorithm must always terminate after a finite number of
steps.
4. Definiteness: Each step of the algorithm must be clear and unambiguous.
5. Effectiveness: Every step must be basic and essential.

6/25 7/25

Example of Algorithm: Finding largest of n numbers Example of Algorithm: Greatest Common Divisor (GCD)
Input: List L = {a1 , a2 , . . . , an } of n positive integers. For e.g. {5, 2, 4, 6, 1, 3}. In mathematics, the Euclidean algorithm, or Euclid’s algorithm, is an effi-
Output: number {x|x ≥ ai , 1 ≤ i ≤ n} and x ∈ L. For e.g. 6. cient method for computing the GCD of two integers (numbers), the largest
number that divides them both without a remainder.
Algorithm 1: max (L) Algorithm 2: max (L) Input: Two n-digit nonnegative integers, m and n.
Output: GCD of m and n.
◦ Step 1: Start. ◦ Step 1: Start.
◦ Step 2: If (|L| == 1) ◦ Step 2: If (|L| == 1)
◦ Step 3: return (a1 ) ◦ Step 3: return (a1 ) Approach 1: Euclid’s Algorithm (m, n) [2] Example illustrating Euclid algorithm

◦ Step 4: Else ◦ Step 4: Else ◦ Step 1: Start. 1. gcd (60, 24)

◦ Step 5: L′ = L − {a1 } ◦ Step 5: Split L into L1 and L2 ◦ Step 2: while n 6= 0 do 2. Here m = 60, and n = 24;
◦ Step 6: y = max (L′ ) ◦ Step 6: x = max (L1 ) ◦ Step 3: r ← m mod n ◮ r ← 60 mod 24 = 12, m ← 24,
◦ Step 7: If (a1 > y ) ◦ Step 7: y = max (L2 ) ◦ Step 4: m ← n n ← 12
◦ Step 8: y = a1 ◦ Step 8: If (x > y ) ◦ Step 5: n ← r 3. Now m = 24, and n = 12;
◦ Step 9: return y. ◦ Step 9: return x. ◦ Step 6: Return m ◮ r ← 24 mod 12 = 0, m ← 12, n
◦ Step 10: Stop. ◦ Step 10: Else ◦ Step 7: Stop. ←0
◦ Step 11: return y. 4. As n = 0, so algorithm will return 12 (m
value).
◦ Step 12: Stop.
8/25 9/25
Example of Algorithm: Greatest Common Divisor (GCD) Example of Algorithm: Greatest Common Divisor (GCD)

Input: Two n-digit nonnegative integers, m and n.

Input: Two n-digit nonnegative integers, m and n. Output: GCD of m and n.
Output: GCD of m and n.

Approach 3: Middle-School Procedure (m, n)[2]

Approach 2: Consecutive integer checking algorithm (m, n) [2]
◦ Step 1: Start.
◦ Step 1: Start.
◦ Step 2: Find the prime factors of m.
◦ Step 2: Assign the value of min{m, n} to t.
◦ Step 3: Find the prime factors of n.
◦ Step 3: Divide m by t. If the remainder of this division is 0, go to
◦ Step 4: Identify all the common factors in the two prime expansions
Step 4; otherwise, go to Step 5.
found in Step 1 and Step 2. (If p is a common factor occurring pm
◦ Step 4: Divide n by t. If the remainder of this division is 0, return and pn times in m and n, respectively, it should be repeated min{pm
the value of t as the answer and stop; otherwise, proceed to Step 5. , pn } times.)
◦ Step 5: Decrease the value of t by 1. Go to Step 3. ◦ Step 5: Compute the product of all the common factors and return
◦ Step 6: Stop. it as the greatest common divisor of the numbers given.
◦ Step 6: Stop.

10/25 11/25

Example of Algorithm: Sorting a set of Natural Numbers Fundamentals of Algorithmic Problem Solving
1
Input: A sequence of n numbers ha1 , a2 , . . . , an i. For e.g. h5, 2, 4, 6, 1, 3i
Output: A permutation (reordering) ha1′ , a2′ , . . . , an′ i of the input sequence such
that a1′ ≤ a2′ ≤ . . . ≤ an′ . For e.g. h1, 2, 3, 4, 5, 6i.

1
Such an input sequence is called an instance of the sorting problem. 12/25 3/11

Fundamentals of Algorithmic Problem Solving. . . Fundamentals of Algorithmic Problem Solving. . .

4. Algorithm design technique or strategy or paradigm
1. Understanding the Problem
- It is a general approach to solving problems algorithmically that is applicable
- Read the problem’s description carefully and ask questions.
to a variety of problems from different area of computing. The various
- Do a few small examples by hand, think about special cases, and ask algorithmic problem solving techniques are:
questions again if needed. ◮ Brute force and exhaustive search
- If while solving any problem there arise a sub-problem for which there ◮ Divide-and-Conquer
already exists an algorithm then it is better to use it. ◮ Dynamic Programming
- It is important to specify exactly the set of instances the algorithm need ◮ Greedy Technique
to handle. 5. Designing an Algorithm and Data Structures
2. Ascertaining the Capabilities of the Computational Device - Designing an algorithm for a particular problem may still be a challenging
task.
- Most of the algorithms in use today are still destined to be programmed
- Some design techniques may be inapplicable to the problem.
for a computer closely resembling the von Neumann machine.
- Even when a particular design technique is applicable, getting an algorithm
- The classic techniques for Design and Analysis of Algorithms are studied is a non-trivial goal.
under the random-access machine (RAM) model.
- Appropriate data structures to be chosen for the operations performed by
3. Choosing between Exact and Approximate Problem Solving the algortihm.
- Choose between solving problem exactly or solving it approximately. 6. Methods of Specifying an Algorithm
◮ Exact algorithm, and - There are two methods to for specifying an algorithm:
◮ Approximation algorithm (why would one opt for this?) ◮ Natural language, and
◮ Psuedocode: Natural language + programming language.
4/11 5/11
Fundamentals of Algorithmic Problem Solving. . . Important Problem Types
7. Proving an Algorithm’s Correctness ◮ Problems are limitless.
- It is to be proved that the algorithm yields a required result for every ◮ The most important problem types are:
legitimate input in a finite amount of time. 1. Sorting: stable and in-place
- A common technique for proving correctness are: mathematical induction, 2. Searching: The searching problem deals with finding a given value,
loop invariant etc. called a search key, in a given set. The e.g. are:
- To prove algorithm is incorrect you need just one instance of input for ◮ Linear search, and
which the algorithm fails. ◮ Binary search.
3. String processing: Searching for a word in a text has attracted spe-
8. Analyzing an Algorithm
cial attention from researchers, it is called as string matching. Sev-
- After correctness by far the most important is efficiency. There are two eral algorithms are developed in this direction:
kinds of algorithm efficiency: ◮ The naive string-matching algorithm,
◮ Time efficiency, and ◮ The Rabin-Karp algorithm,
◮ Space efficiency ◮ The Knuth-Morris-Pratt algorithm, and
- Other desirable characteristics of an algorithm are: ◮ Boyer Moore algorithm.
◮ Simplicity 4. Graph problems: Graphs can be used for modeling a wide variety
◮ Generality of applications, such as transportation, communication, social and
economic network etc.
9. Coding an Algorithm 5. Geometric problems: It deals with geometric objects such as points,
- Most algorithms are destined to be ultimately implemented as computer lines, and polygon for e.g. closest-pair problem.
programs. 6. Numerical problems: It involves solving equations and system of
- Of course implementing an algorithm correctly is necessary but not suffi- equations, computing definite integrals, evaluating functions, and so
cient. 6/11 on. 7/11

Fundamentals of Algorithmic Problem Solving The Analysis Framework

1. Measuring an input size
2. Running time [1]:
• The running time of an algorithm on a particular input is the number of
primitive operations or “steps” executed.

What is meant by analyzing algorithms?

1. It involves thinking about how their resource requirements − the amount
of time and space they use − will scale with increasing input size.
2. There are two kinds of efficiency:
◮ Time efficiency (or Time complexity): How fast an algorithm runs?
◮ Space efficiency (or Space complexity): It refers to the amount of
memory units required by the algorithm in addition to space needed
for its input and output.

• The running time of the algorithm is the sum of running times for each
statement executed.

8/11 3/7

The Analysis Framework. . . The Analysis Framework. . .

1. Orders of growth/Rate of growth: We will see the order of growth of 1. Best-Case, Worst-Case, and Average-Case Efficiencies. . .:
functions with larger input size. The growth of some of the functions is • Worst-Case: If the array is in reverse sorted order−that is, in decreasing
shown in table below [2]: order−the worst case results. So,

So, the running time of Insertion-Sort can be expressed as an2 + bn + c.

It is thus a quadratic function in n. Here,
2. Best-Case, Worst-Case, and Average-Case Efficiencies:
• Best-Case: In case of Insertion-Sort, the best case occurs when the input
array is already sorted. For best case tj will be 1. So,

So, the running time of Insertion-Sort can be expressed as an + b. It is

• Average-Case: The “average case” is often roughly as bad as the worst
thus a linear function in n. 4/7 case. 5/7
Asymtotic notations Θ notation
We denote by Θ(g (n)) the set of functions
1. The notations we use to describe the asymptotic running time of an al-
gorithm are defined in terms of functions whose domains are the set of
natural numbers N = {0, 1, 2, . . .}.
2. Such notations are convenient for describing the worst-case running-time
function T (n), which usually is defined only on integer input sizes.
3. Asymptotic notation actually applies to functions. ◮ As Θ(g (n)) is a set, we could
4. For example the worst-case running time of Insertion-Sort is an2 + bn + c, write “f (n) ∈ Θ(g (n))”. It
for some constants a, b, and c. We can write the worst-case running time means f (n) is a member of
of Insertion-Sort as Θ(n2 ). Θ(g (n)).
5. To compare and rank orders of growth, computer scientists use three no- ◮ g (n) is asymptotically tight
tations: bound for f (n).
◮ Θ (Big theta), ◮ The above definition re-
◮ Ω (Big omega), and quires that every member
◮ O (Big oh) f (n) ∈ Θ(g (n)) be asymptot-
ically nonnegative, that is,
f (n) be nonnegative whenever
n is sufficiently large.
3/9 4/9

O notation Ω notation
We denote by O(g (n)) (pronounced “big-oh of g of n” or somtimes We denote by Ω(g (n)) (pronounced “big-omega of g of n” or som-
just “oh of g of n”) the set of functions times just “omega of g of n”) the set of functions

◮ We use O−notation to have ◮ We use Ω−notation to have

only an asymptotic upper only an asymptotic lower
bound. bound.
◮ Intution: For all values n at ◮ Intution: For all values n at
and to the right of n0 , the value and to the right of n0 , the value
of the function f (n) is on or be- of the function f (n) is on or
low cg (n). above cg (n).
◮ f (n) = O(g (n)) indicate that ◮ f (n) = Ω(g (n)) indicate that
a function f (n) is a member of a function f (n) is a member of
the set O(g (n)). the set Ω(g (n)).

5/9 6/9

o notation ω notation

• We formally define o(g (n)) (pronounced “little-oh of g of n” as • We formally define ω(g (n)) (pronounced “little-ω of g of n” as
the set the set

• Some authors use this limit as a definition of the o-notation • Some authors use this limit as a definition of the ω-notation

f (n) f (n)
lim =0 lim =∞
n→∞ g (n) n→∞ g (n)

8/13 9/13
Relational properties Relational properties. . .
• Assume that f (n) and g (n) are asymptotically positive.
1. Transitivity:

• Assume that f (n) and g (n) are asymptotically positive.

1. Transpose symmetry:

2. Reflexivity:

3. Symmetry:

10/13 11/13
Mathematical Analysis of
Non-recursive Algorithms
 We systematically apply the general
framework (discussed earlier) to
analyzing the time efficiency of non-
recursive algorithms.
Mathematical Analysis of
Non-recursive Algorithms
General Plan for Analyzing the Time Efficiency
 4. Set up a sum expressing the number of times the
algorithm’s basic operation is executed.
 5. Using standard formulas and rules of sum manipulation,
either find a closed-form formula for the count or, at the
very least, establish its order of growth.
Mathematical Analysis of
Non-recursive Algorithms
 input’s size here is the number of elements in the
array, i.e., n.
Mathematical Analysis of
Non-recursive Algorithms
General Plan for Analyzing the Time Efficiency
 1. Decide on a parameter (or parameters) indicating an
input’s size.
 2. Identify the algorithm’s basic operation. (As a rule, it
is located in the innermost loop.)
 3. Check whether the number of times the basic
operation is executed depends only on the size of an
input. If it also depends on some additional property, the
worst-case, average-case, and, if necessary, best-case
efficiencies have to be investigated separately.
Mathematical Analysis of
Non-recursive Algorithms
Example 1
 Consider the problem of finding the value of
the largest element in a list of n numbers.
 For simplicity, we assume that the list is
implemented as an array.
Mathematical Analysis of
Non-recursive Algorithms
 Algorithm’s for loop-
 There are two operations in the loop’s body: the
comparison A[i]> maxval and
 the assignment maxval←A[i].
 Which of these two operations should we
consider basic?
Mathematical Analysis of
Non-recursive Algorithms
 Since the comparison is executed on each
repetition of the loop and the assignment is not,
we should consider the comparison to be the
algorithm’s basic operation.
Mathematical Analysis of
Non-recursive Algorithms
 Let us denote C(n) the number of times this comparison is
executed.
 The algorithm makes one comparison on each execution of
the loop, which is repeated for each value of the loop’s
variable i within the bounds 1 and n − 1, inclusive.
 Therefore, we get the following sum for C(n):

 Note that the number of comparisons will be the

same for all arrays of size n; therefore, in terms of
this metric, there is no need to distinguish among
the worst, average, and best cases here.

Mathematical Analysis of Mathematical Analysis of

Non-recursive Algorithms Non-recursive Algorithms
EXAMPLE 2

 Consider the element uniqueness problem:

check whether all the elements in a given
array of n elements are distinct.

 This problem can be solved by the following  input’s size here is again n, the number of
straightforward algorithm. elements in the array.

Mathematical Analysis of Mathematical Analysis of

Non-recursive Algorithms Non-recursive Algorithms
 Since the innermost loop contains a single
operation (the comparison of two elements), we
should consider it as the algorithm’s basic
operation.
 Note, however, that the number of comparisons
depends not only on n but also on whether there
are equal elements in the array and, if there are,
which array positions they occupy.
 Let us investigate the worst case scenario.
Mathematical Analysis of Mathematical Analysis of
Non-recursive Algorithms Recursive Algorithms

Some formula to consider

◼ We systematically apply the general
framework (discussed earlier) to
analyzing the time efficiency of
recursive algorithms.

Mathematical Analysis of Mathematical Analysis of

Recursive Algorithms Recursive Algorithms
General Plan for Analyzing the Time Efficiency General Plan for Analyzing the Time Efficiency

◼ 1. Decide on a parameter (or parameters) ◼ 4. Set up a recurrence relation, with an

indicating an input’s size. appropriate initial condition, for the number of
◼ 2. Identify the algorithm’s basic operation. times the basic operation is executed.

◼ 3. Check whether the number of times the basic ◼ 5. Solve the recurrence or, at least, ascertain the
operation is executed can vary on different inputs order of growth of its solution.
of the same size; if it can, the worst-case,
average-case, and best-case efficiencies must
be investigated separately.

Mathematical Analysis of Mathematical Analysis of

Recursive Algorithms Recursive Algorithms

Example 1
◼ Compute the factorial function F(n) = n! for
an arbitrary nonnegative integer n.

Since n!= 1 . . . . . (n − 1) . n = (n − 1)! . n for n ≥

1 and 0!= 1 by definition, we can compute F(n) =
For simplicity, we consider n as an indicator of this
F(n − 1) . n with the following recursive algorithm.
◼

algorithm’s input size.

Mathematical Analysis of Mathematical Analysis of
Recursive Algorithms Recursive Algorithms
◼ The basic operation of the algorithm is
◼ This equation defines M(n) not explicitly, i.e.,
multiplication, whose number of executions we
as a function of n, but implicitly as a function of
denote M(n).
its value at another point, namely n−1.
◼ Since the function F(n) is computed according to
the formula F(n) = F(n − 1) . n for n > 0, the ◼ Such equations are called recurrence relations
number of multiplications M(n) needed to or recurrences.
compute it must satisfy the equality
◼ Our goal now is to solve the recurrence
relation M(n) = M(n − 1) + 1, i.e., to find an
explicit formula for M(n) in terms of n only.

Mathematical Analysis of Mathematical Analysis of

Recursive Algorithms
◼ To determine a solution uniquely, we need an
initial condition that tells us the value with
which the sequence starts.
◼ We can obtain this value by inspecting the
condition that makes the algorithm stop its
recursive calls:
if n == 0 return 1.
Recursive Algorithms
◼ First, since the calls stop when n = 0, the
smallest value of n for which this algorithm is
executed and hence M(n) defined is 0.
◼ Second, by inspecting the pseudocode’s exiting
line, we can see that when n = 0, the algorithm
performs no multiplications.
◼ Therefore, the initial condition is:

Mathematical Analysis of Mathematical Analysis of

Recursive Algorithms Recursive Algorithms
◼ Thus, the recurrence relation and initial condition ◼ Solution
for the algorithm’s number of multiplications M(n):
M(n) = M(n − 1) + 1 for n > 0,
0 for n=0.

◼ Of the general form M(n) = M(n − i) + i

◼ We use the method of backward substitutions M(n) = M(n − 1) + 1 = . . .
to solve this recurrence relation.
= M(n − i) + i = . . .
= M(n − n) + n = n.
Mathematical Analysis of Mathematical Analysis of
Recursive Algorithms Recursive Algorithms
General Plan for Analyzing the Time Efficiency General Plan for Analyzing the Time Efficiency

◼ 1. Decide on a parameter (or parameters) ◼ 4. Set up a recurrence relation, with an

indicating an input’s size. appropriate initial condition, for the number of
◼ 2. Identify the algorithm’s basic operation. times the basic operation is executed.

◼ 3. Check whether the number of times the basic ◼ 5. Solve the recurrence or, at least, ascertain the
operation is executed can vary on different inputs order of growth of its solution.
of the same size; if it can, the worst-case,
average-case, and best-case efficiencies must
be investigated separately.

Mathematical Analysis of Mathematical Analysis of

Recursive Algorithms Recursive Algorithms
Tower of Hanoi
Example 2
◼ In this puzzle, we have n disks of different sizes
◼ Solving the Tower of Hanoi puzzle. that can slide onto any of three pegs.
◼ Initially, all the disks are on the first peg in order
of size, the largest on the bottom and the smallest
on top.
◼ The goal is to move all the disks to the third peg,
using the second one as an auxiliary, if necessary.
◼ We can move only one disk at a time, and it is
forbidden to place a larger disk on top of a smaller
one.

Mathematical Analysis of Mathematical Analysis of

Recursive Algorithms Recursive Algorithms
Recursive solution to the Tower of Hanoi puzzle
◼ To move n>1 disks from peg 1 to peg 3 (with peg 2 as
auxiliary), we first move recursively n − 1 disks from peg
1 to peg 2 (with peg 3 as auxiliary), then move the largest
disk directly from peg 1 to peg 3, and, finally, move
recursively n − 1 disks from peg 2 to peg 3 (using peg 1 as
auxiliary).

◼ Of course, if n = 1, we simply move the single disk directly

from the source peg to the destination peg.
Recursive solution to the Tower of Hanoi puzzle
Mathematical Analysis of Mathematical Analysis of
Recursive Algorithms Recursive Algorithms
Recursive solution to the Tower of Hanoi puzzle Recursive solution to the Tower of Hanoi puzzle
◼ The input’s size indicator- the number of disks n
◼ Basic operation- moving one disk
◼ The number of moves M(n) depends on n only,
and the recurrence equation is the following: after i substitutions, we get

◼ With the initial condition, we get the recurrence for n = 1, which is achieved for i = n − 1, we
relation as

Mathematical Analysis of Mathematical Analysis of

Recursive Algorithms
Recursive solution to the Tower of Hanoi puzzle
◼ When a recursive algorithm makes more than a single call
to itself, it can be useful for analysis purposes to construct
a tree of its recursive calls.
Recursive Algorithms
Recursive solution to the Tower of Hanoi puzzle
◼ By counting the number of nodes in the tree, we can get
the total number of calls made by the Tower of Hanoi
algorithm:

Mathematical Analysis of Mathematical Analysis of

Recursive Algorithms Recursive Algorithms
Binary number representation.
Example 3
◼ Recurrence relation
◼ Binary number representation.

◼ Standard approach to solving such a recurrence is

to solve it only for n = 2k and then take
advantage of the theorem called the smoothness
rule.
Mathematical Analysis of Travelling Salesman Problem
Recursive Algorithms
◼ Recurrence relation Solution ◼ The traveling salesman problem (TSP)
has been an interesting problem to
researchers for the last 150 years by
◼ its seemingly simple formulation
◼ important applications
◼ planning, scheduling, logistics and packing

◼ interesting connections to other combinatorial

problems

Travelling Salesman Problem Travelling Salesman Problem

◼ The problem asks to find the shortest tour ◼ Then the problem can be stated as the
through a given set of n cities that visits
problem of finding the shortest
each city exactly once before returning to
Hamiltonian circuit/cycle of the graph.
the city where it started.
◼ The problem can be conveniently modeled
by a weighted graph, with the graph’s ◼ A Hamiltonian circuit/cycle is defined as a
vertices representing the cities and the edge cycle that passes through all the vertices of
weights specifying the distances. the graph exactly once.

Travelling Salesman Problem Travelling Salesman Problem

◼ A Hamiltonian circuit can also be defined as a

sequence of n + 1 adjacent vertices

where the first vertex of the sequence is

the same as the last one and all the other

Examples: Hamiltonian circuit/cycle n − 1 vertices are distinct.

Travelling Salesman Problem Travelling Salesman Problem

◼ Further, we can assume, that all circuits start Example:

and end at one particular vertex (cycles).

◼ Thus, we can get all the tours by

◼ generating all the permutations of n − 1
intermediate cities,
◼ compute the tour lengths, and
◼ find the shortest among them.

Travelling Salesman Problem Travelling Salesman Problem

Solution: ◼ Three pairs of tours that differ only by their

direction.
◼ Hence, we could cut the number of vertex
permutations by half.
◼ We could, for example, choose any two
intermediate vertices, say, b and c, and then
consider only permutations in which b
precedes c. (This trick implicitly defines a
tour’s direction.)

Travelling Salesman Problem Knapsack Problem

◼ Given n items of known weights w1, w2, ...,

◼ The total number of permutations wn and values v1, v2, ..., vn and a knapsack
needed is still of capacity W, find the most valuable subset
of the items that fit into the knapsack.
◼ A thief who wants to steal the most valuable
which makes the exhaustive-search loot that fits into his knapsack

approach impractical for all but very small ◼ A transport plane that has to deliver the
most valuable set of items to a remote
values of n. location without exceeding the plane’s capacity.
Knapsack Problem- Types Knapsack Problem- Types

◼ 0/1 Knapsack Problem ◼ Fractional Knapsack Problem

◼ Given a set of n items numbered from 1 up to ◼ Given a set of n items numbered from 1 up to
n, each with a weight wi and a value vi, along n, each with a weight wi and a value vi, along
with a maximum weight capacity W, with a maximum weight capacity W,

where xi represents the number of instances where xi R represents the whole/fraction of

of item i to include in the knapsack. instances of item i to include in the knapsack.

0/1 Knapsack Problem 0/1 Knapsack Problem

◼ Brute-Force/Exhaustive Solution ◼ Brute-Force/Exhaustive Solution

◼ Generating all the subsets of the set of n ◼ Since the number of subsets of an n-element
items given, set is 2n, the exhaustive search leads to an
◼ Computing the total weight of each subset algorithm of order
in order to identify feasible subsets (i.e., the
ones with the total weight not exceeding the
knapsack capacity), and
no matter how efficiently individual subsets are
◼ Finding a subset of the largest value among generated.
them.

0/1 Knapsack Problem 0/1 Knapsack Problem

Example: Example:
NP-Hard Problems
◼ Thus, for both the traveling salesman and
knapsack problems considered above,
exhaustive search leads to algorithms that are
extremely inefficient on every input.
◼ In fact, these two problems are the best-known
examples of so called NP-hard problems.
[Non-deterministic Polynomial-time Hard]
◼ No polynomial-time algorithm is known for any
NP-hard problem.
Assignment Problem
Example
◼ A taxi firm (Ola/Uber) has three taxis (agents)
available, and three customers (tasks) wishing to
be picked up as soon as possible.
◼ The firm prides itself on speedy pickups, so for
each taxi the "cost" of picking up a particular
customer will depend on the time taken for the
taxi to reach the pickup point.
◼ The solution to the assignment problem will be
whichever combination of taxis and customers
results in the least total cost.
Assignment Problem
◼ It is easy to see that an instance of the
assignment problem is completely specified
by its cost matrix C.
◼ In terms of this matrix, the problem is to
select one element in each row of the
matrix so that all selected elements are in
different columns and the total sum of the
selected elements is the smallest possible.
◼ Note that no obvious strategy for finding a
solution works here.
Assignment Problem
◼ There are n people (agents) who need to
be assigned to execute n tasks, one person
per job. (That is, each person is assigned to
exactly one job and each job is assigned to
exactly one person.)
◼ The cost that would accrue if the ith person
is assigned to the jth job is a known
quantity C[i,j] for each pair i,j = 1, 2, …, n.
◼ The problem is to find an assignment with
the minimum total cost.
Assignment Problem
◼ A small instance of this problem follows,
with the table entries representing the
assignment costs C[i,j]:
Assignment Problem
◼ For example, we cannot select the
smallest element in each row, because the
smallest elements may happen to be in the
same column.
◼ In fact, the smallest element in the entire
matrix need not be a component of an
optimal solution.
◼ Thus, opting for the exhaustive search may
appear as an unavoidable evil.
Assignment Problem Assignment Problem

Brute-Force Approach Brute-Force Approach

◼ We can describe feasible solutions to the ◼ For example, for the cost matrix (below),
<2,3,4,1> indicates: ???
assignment problem as n-tuples <j1, …, jn>
◼ the assignment of Person 1 to Job 2, Person 2 to
in which the ith component, i = 1, …, n, Job 3, Person 3 to Job 4, and Person 4 to Job 1.
indicates the column of the element
selected in the ith row (i.e., the job number
assigned to the ith person).

Assignment Problem Assignment Problem

Brute-Force Approach Brute-Force Approach

◼ There should be a one-to-one correspondence
between feasible assignments and
permutations of the first n integers.

1. generating all the permutations of integers 1,2,…,

n;
2. computing the total cost of each assignment by
summing up the corresponding elements of the
cost matrix;
3. finally selecting the one with the smallest sum.

Assignment Problem

Brute-Force Approach
◼ Since the number of permutations to be
considered for the general case of the
assignment problem is

◼ exhaustive search is impractical for all but

very small instances of the problem.
◼ Fortunately, there is a much more efficient
algorithm for this problem called the
Hungarian method.
 The Divide-and-Conquer approach

IDEA:
1. Break the problem into several subproblems that are similar to the original
problem but smaller in size,
2. Solve the subproblems recursively, and
3. then combine these solutions to create a solution to the original problem.

The divide-and-conquer paradigm involves three steps at each level of the recursion:

• Divide the problem into a number of subproblems that are smaller instances of
the same problem.
• Conquer the subproblems by solving them recursively. If the subproblem sizes
are small enough, however, just solve the subproblems in a straightforward
manner.
• Combine the solutions to the subproblems into the solution for the original
problem.

MERGE SORT:

Input: a sequence of n numbers <a1, a2, ..., an>.

Output: A permutation (reordering ) <a1’, a2’, ..., an’> of the input sequence such
that a1’<= a2’<=...<= an’.

The merge sort algorithm closely follows the divide-and-conquer paradigm.

Intuitively, it operates as follows.

• Divide: Divide the n-element sequence to be sorted into two subsequences of

n=2 elements each.
• Conquer: Sort the two subsequences recursively using merge sort.
• Combine: Merge the two sorted subsequences to produce the sorted answer.

ALGORITHM:

EXAMPLE:

ANALYSIS OF MERGE SORT:

We can also write;