Complexity Theory

Types of Problems
 Simple problems : Problems that can be solved in
Polynomial Time; f(n)  Θ(nk), where n is input
size & k any positive integer
 Example:
Job Sequencing Prim, Kruskal
Knapsack Dijkstra, Bellman Ford
Matrix Chain Floyd Warshall
BFS, DFS Network Flow

And many others

 Types of Problems
 Complex problems : Problems that can be NOT
solved in Polynomial Time
 Why are they important?

“These problems capture vast domains of

computational, scientific and mathematical
problems, and seems to delimit what scientists and
mathematicians have been aspiring to compute
feasibly”
 Simple to Complex
 Some familiar problems become complex with
small changes:
 0-1 Knapsack
• Simple when item weights are in integer
• Complex when items weights are real numbers
 Graph
• Simple for Minimum distance problem
• Complex for Maximum distance
 More
• Finding an Euler Path in a graph is Simple
• Finding a Hamiltonian Path is not
 Complex Problems

More examples:
• Satisfiability problem
• Vertex Cover
• Independence Set
• Set cover
• Clique
• Subset Sum
• Knapsack
 Types of Problems
 Decision Problems
Solution exist ? (Yes/No)
 Search Problems
Any valid solution
 Optimization Problems
Optimum solution only

Complexity considerations based on only

Decision Type of Problems
 Complexity Classes
 P Class:
Problems solved in Polynomial time, thus, an
algorithm solving the problem has total time
in polynomial order of input size of problem

 NP Class:
Problems verified in Polynomial time, given a
solution to the problem
 Complexity Classes
 P Class:
P class problems are solved in polynomial
time, they are also verifiable in polynomial
time

 NP Class:
Solution not possible in polynomial time, but
any given solution can be verified in
polynomial time
 P & NP
 Relationship between two classes
P ⊆ NP
 Reasoning:
P & NP problems are both verifiable in polynomial
time, hence NP class contains all problems in P
class, along with problems not solvable in
polynomial time
 P & NP
 Relationship between two classes
P ⊆ NP
 Reasoning:
P & NP problems are both verifiable in polynomial
time, hence NP class contains all problems in P
class, along with problems not solvable in
polynomial time

This relationship is generally accepted,

but not yet proven mathematically
P & NP

NP Class

P Class
 P & NP

NP Class

P Class

Major Research Topic : P = NP ?

 More Definition
 Reduction:
Conversion of a Decision Problem A is to
another Decision Problem B, in polynomial
time, such that any solution of B will be
applicable for A.
 More Definition
 Reduction:
Conversion of a Decision Problem A is to
another Decision Problem B, in polynomial
time, such that any solution of B will be
applicable for A.

Notation: A ≤P B
 Reduction Process
Reducing Decision Problem A to B:
• A Polynomial time algorithm translates all inputs
of A to B
• If the input results in Yes output in A, it should
result in Yes output in B
• If the input results in No output in A, it should
result in No output in B
Yes Yes
Polynomial-time
reduction from A to B Algorithm to solve B
Input to A Input to No No
B

Problem A is Reduced to Problem B

 Properties of Reduction
 A ≤P B
If A can be reduced to B, it implies they belong to
same class of problems
Thus,
If A ∈ P Class, then B ∈ P Class
If A ∈ NP Class, then B ∈ NP Class
 Properties of Reduction
 A ≤P B
If A can be reduced to B, it implies they belong to
same class of problems
Thus,
If A ∈ P Class, then B ∈ P Class
If A ∈ NP Class, then B ∈ NP Class

 If A ≤P B and B ≤P C, then A ≤P C
(Transitive Property)
 More Definitions
 NP Hard
If a known NP-Hard problem, A, can be
Reduced in polynomial time to another
problem, B, then B is also NP-Hard
 More Definitions
 NP Hard
If a known NP-Hard problem, A, can be
Reduced in polynomial time to another
problem, B, then B is also NP-Hard

All NP-Hard problems are equivalent; if a

polynomial time solution exists for one, similar
solution can be derived for others
 More Definitions
 NP Complete
A Problem is NP-Complete if
– It is NP-Hard, and
– It is NP
 More Definitions
 NP Complete
A Problem is NP-Complete if
– It is NP-Hard, and
– It is NP

NP-Complete problems can be reduced from other

similar problems and can be verified in
polynomial time
P, NP, NP-Hard & NP-Complete

NP NP Hard

NP
Complete
 Satisfiability Problem
 Φ is a Boolean expression, written in Conjunctive
Normal Form (CNF) as :
Φ = (x1 ∨ x2 V x3 V x4 ) ∧ (x1 ∨ x5 V x6 V x4 ) ∧
(x5 ∨ x6 V x7) ∧ (x1 ∨ x7 V x8 V x4 V x9 ) …
x1, x2, x3, x4, … - Boolean Variables (True/False)
 Satisfiability Problem
 Φ is a Boolean expression, written in Conjunctive
Normal Form (CNF) as :
Φ = (x1 ∨ x2 V x3 V x4 ) ∧ (x1 ∨ x5 V x6 V x4 ) ∧
(x5 ∨ x6 V x7) ∧ (x1 ∨ x7 V x8 V x4 V x9 ) …
x1, x2, x3, x4, … - Boolean Variables (True/False)

Satisfiability Problem (SAT):

(Satisfying Truth Assignment)
For what value of variables x1, x2, x3, x4, x5 …,
is Φ TRUE?
 Cook’s Theorem

 Cook’s Theorem:
Satisfiability Problem is NP

Proven that no algorithm exists to solve

Satisfiability Problem in polynomial time.
But, given a solution, it can be verified in
polynomial time.
 Cook’s Theorem

 Cook’s Theorem:
Satisfiability Problem is NP

Proven that no algorithm exists to solve

Satisfiability Problem in polynomial time.
But, given a solution, it can be verified in
polynomial time.

The most fundamental problem in NP

Problem Hierarchy
 3-CNF Satisfiability

 Boolean expression containing 3 variables

per group:
Φ = (x1 ∨ x2 V x3 ) ∧ (x5 V x6 V x4 ) ∧ (x5 ∨ x6
∨ x4 ) ∧ (x6 V x8 V x4 ) ….
Total number of variables and groups per
expression not limited

Any Boolean expression can be Reduced to

3-CNF format in polynomial time
 Clique
 A Clique is a complete subgraph in a graph
A C

S T

B D

 Vertices in a Clique are connected to others

 Size of a Clique = number of vertices
 Maximum Clique
 A Clique is a complete subgraph in a graph
A C

S T

B D

 Problem: Find the Clique of maximum size

 NP-Complete problem
 Clique Decision Problem (CDP)

A C

S T

B D

 Input: A Graph (G) and number (k)

 Output: Clique of size k exists in G? (Yes / No)
 Prove that CDP is NP-Complete
 Reducing 3-CNF to CDP

 3-CNF expression:
Φ = (x1 V x2 V x3 ) ∧ (x1 V x2 V x3 ) ∧ (x2
V x1 V x3 ) ∧ (x1 V x2 V x3 ) ….

Number of variables : 3
Number of groups: k (as in CDP)

Construct a Graph from this expression

 Reducing 3-CNF to CDP

 Graph Construction:
• Assume k = 3
• Each Boolean variable maps to a vertex in
graph
• Vertices connected via edges, except for :
– Variables in the same group(within same
clauses)
– Variables in negation, e.g., x3 not to be
connected to x3
 Reducing 3-CNF to CDP

 Example:
Φ = (x1 v x2 v x3 ) ∧ (x1 v x2 v x3 ) ∧ (x1 v x2 v x3 )

X1 X2 X3

X1 X1
X2 X2

X3 X3
 Reducing 3-CNF to CDP

 Example:
Φ = (x1 v x2 v x3 ) ∧ (x1 v x2 v x3 ) ∧ (x1 v x2 v x3 )

X1 X2 X3

X1 X1
X2 X2

X3 X3
 Reducing 3-CNF to CDP

 Example:
Φ = (x1 v x2 v x3 ) ∧ (x1 v x2 v x3 ) ∧ (x1 v x2 v x3 )

X1 X2 X3

X1 X1
X2 X2

X3 X3
 Reducing 3-CNF to CDP

 Example:
Φ = (x1 v x2 v x3 ) ∧ (x1 v x2 v x3 ) ∧ (x1 v x2 v x3 )

X1 X2 X3

X1 X1
X2 X2

X3 X3
 Reducing 3-CNF to CDP

 Example:
Φ = (x1 v x2 v x3 ) ∧ (x1 v x2 v x3 ) ∧ (x1 v x2 v x3 )

X1 X2 X3

X1 X1
X2 X2

X3 X3
 Reducing 3-CNF to CDP

 Mapping:
Φ = (x1 v x2 v x3 ) ∧ (x1 v x2 v x3 ) ∧ (x1 v x2 v x3 )
• Φ is satisfied when :
– x1 is TRUE, x2 is TRUE and x3 is TRUE
– x1 is FALSE, x2 is TRUE and x3 is FALSE
– x1 is TRUE, x2 is FALSE and x3 is TRUE
– And others
 Reducing 3-CNF to CDP

 Mapping:
Φ = (x1 v x2 v x3 ) ∧ (x1 v x2 v x3 ) ∧ (x1 v x2 v x3 )
• Φ is satisfied when :
– x1 is TRUE, x2 is TRUE and x3 is TRUE
– x1 is FALSE, x2 is TRUE and x3 is FALSE
– x1 is TRUE, x2 is FALSE and x3 is TRUE
– And others
• A Clique of size k exists in the graph, with
vertices matching the condition that satisfies
3-CNF expression
 Reducing 3-CNF to CDP

 Reduction Process:
• Conversion of 3-CNF Boolean expression to a
Graph can be completed in polynomial time
• For all inputs that satisfy the Boolean expression,
Cliques exist in the graph
• For all inputs that do not satisfy the Boolean
expression, Cliques do not exist

Current example uses k=3, which can be changed to any

value following the same procedure
 Reducing 3-CNF to CDP

 Reduction Process:
• Conversion of 3-CNF Boolean expression to a
Graph can be completed in polynomial time
• For all inputs that satisfy the Boolean expression,
Cliques exist in the graph
• For all inputs that do not satisfy the Boolean
expression, Cliques do not exist

3-CNF ≤P CDP
 Clique Decision Problem (CDP)
 Clique Decision Problem is NP-
Complete:
• A known NP-Hard problem, 3-CNF, is reduced
to CDP in polynomial time
• Any given solution for CDP can be verified in
polynomial Time
 Clique Decision Problem (CDP)
 Clique Decision Problem is NP-
Complete:
• A known NP-Hard problem, 3-CNF, is reduced
to CDP in polynomial time
• Any given solution for CDP can be verified in
polynomial Time

 NP Completeness Proof of any problems

must include these points
 Space Complexity
 P Class:
Decision problems solved in Polynomial time
 PSPACE Class:
Decision problem solved in Polynomial Space.
The space requirement for algorithm is a
polynomial multiple of input size of problem.
 Space Complexity
 P Class:
Decision problems solved in Polynomial time.
 PSPACE Class:
Decision problem solved in Polynomial Space.

Relation: P ⊆ PSPACE
Problems solved in polynomial time can use only
polynomial space
 Space Complexity
 Claim: 3–CNF ∈ PSPACE
Reasoning:
Evaluate all possibilities of n variables (2n) for a
CNF using a n bit binary counter, which only
requires n bits of space
 Space Complexity
 Claim: 3–CNF ∈ PSPACE
Evaluate all possibilities of n variables (2n) for a
CNF using a n bit binary counter, which only
requires n bits of space

 NP ⊆ PSPACE
Any problem in NP can be reduced from 3-CNF,
hence all NP problems belong to PSPACE
 P, NP & PSPACE

PSPACE

NP
 PSPACE Complete
 PSPACE Complete
A Problem, B, is PSPACE-Complete if
– If a Problem A, belonging to PSPACE, can
be reduced to B
– B belongs to PSPACE

Similar definition as NP-Complete

 PSPACE Complexity
 Examples
• Rubik’s Cube
• Planning problem
• Two person game, like chess
• 8-Puzzle or 15-Puzzle
The End