Computational Complexity

P, EXP, R, NP, NP Hard, NP Complete

Outline

 Decision and Optimization Problems

 P , EXP, R and NP
 Polynomial-Time Reducibility
 NP-Hardness and NP-Completeness
Complexity of Problems

 We have seen that decision problems (and their associated languages)

can be classified into decidable and un decidable.

Decidable
Languages
“Easy” Problems
“Hard” Problems
Time Complexity
 The most critical computational resource is often time, so the most
useful complexity measure is often time complexity
 The time complexity of an algorithm quantifies the amount of time
taken by an algorithm to run as a function of the length of the input.
 Note that the time to run is a function of the length of the input and
not the actual execution time of the machine on which the algorithm
is running on.
 If we take Turing Machine as our model of computation, then we can
give a precise measure of the time resources used by a computation

Definition
Definition The
The time
timecomplexity
complexity ofofaaTuring
TuringMachine
Machine TT isisthe
the
function such that Time T ( x ) isisthe
suchthat thenumber
numberofof
Time
function T

steps
stepstaken
takenby
bythe
thecomputation
computation T(x)
T(x)
 Space Complexity
 Another important computational resource is amount of “memory”
used by an algorithm, that is space. The corresponding complexity
measure is space complexity
 Problem-solving using computer requires memory to hold temporary
data or final result while the program is in execution.
 The amount of memory required by the algorithm to solve given
problem is called space complexity of the algorithm.
 The space complexity of an algorithm quantifies the amount of space
taken by an algorithm to run as a function of the length of the input.
 As with time, if we take Turing Machine as our model of
computation, then we can easily give a measure of the space resources
used by a computation
Definition
Definition The The space
spacecomplexity
complexity ofofaaTuring
TuringMachine
Machine TT isisthe
the
function Space T such
function suchthat isisthe
thenumber
numberofof
Space ( x )
that T

distinct
distincttape
tapecells
cellsvisited
visitedduring
duringthe
thecomputation
computation T(x)
T(x)
Complexity Classes

A complexity class is the set of all of the computational problems

which can be solved using a certain amount of a certain
computational resource.
 Decision and Optimization Problems
 Decision Problem: computational problem with intended output
of “yes” or “no”, 1 or 0
 A decision problem is a general question to be answered,
usually possessing several parameters, or free variables, whose
values are left unspecified. An instance of a problem is obtained
by specifying particular values for all of the problem
parameters.
 A solution to a decision problem is an algorithm that answers
the question that results from each instance. A decision problem
is decidable if a solution exists and is un decidable otherwise.
 We will be concerned with the question of whether certain
decision problems are decidable. For those that are, in general,
computer science is concerned with finding efficient solutions
Con..

 Optimization Problem: computational problem where we try

to maximize or minimize some value
 Introduce parameter k and ask if the optimal value for the
problem is a most or at least k. Turn optimization into
decision
Class of the problem
 Complexity Class p
 The class P consists of all decision problems that are solvable by
Turing machines in polynomial time.
 We say that problems in P are efficiently solvable.
 The class P consists of those problems that are solvable in
polynomial time.
 More specifically, they are problems that can be solved in time
O(nk) for some constant k, where n is the size of the input to the
problem
 The key is that n is the size of input
Features:
 The solution to P problems is easy to find.
 P is often a class of computational problems that are solvable and
tractable. Tractable means that the problems can be solved in theory
as well as in practice. But the problems that can be solved in theory
but not in practice are known as intractable.
Complexity Class P

 It is Deterministic in nature
 Solved by conventional computers in polynomial time
O(1) Constant
O(log n) Sub-linear
O(n) Linear
O(n log n) Nearly Linear
O(n2) Quadratic
 Consists Polynomial upper and lower bounds
Polynomial Equivalence of RAM algorithms and
Turing Machine algorithms

 A multi tape TM can simulate a RAM algorithm that runs for

time O(T(n)) in at most O(T2(n)) of its own steps.
 If T(n) is a polynomial, so is T2(n).
Examples of Problems in P
 Is w in L(G), for a given CFG G?
 Input = w.
 Use CYK algorithm, which is O(n3).
 Is there a path from node x to node y in graph G?
 Input = x, y, and G.
 Use BFS algorithm, which is O(n) on a graph of n nodes
and arcs.
Examples
Running Times Between Polynomials

 You might worry that something like O(n log n) is not a

polynomial.
 However, to be in P, a problem only needs an algorithm that
runs in time less than some polynomial.
 O(n log n) is less than O(n2).
 So, for most algorithms counting input size in bits or words
is polynomial either way.
EXP, R

 Problems solvable in exponential time. E.g. n*n chess

 Problems solvable in finite time. Halting problems

P EXP R
 NP
 Complexity class NP will serve as our formal model for the class of
problems having efficiently verifiable solutions.
 A decision problem / language is in NP if given an input x, we can
easily verify that x is a YES instance of the problem (or equivalently,
x is in the language) if we are given the polynomial-size solution for
x, that certifies this fact.
 NP is not the same as non-polynomial complexity/running time.
 NP does not stand for not polynomial.
 NP = Non-Deterministic polynomial time
 NP means verifiable in polynomial time
 Verifiable?
 If we are some how given a ‘certificate’ of a solution we can verify
the legitimacy in polynomial time
What happened to automata?

 Problem is in NP iff it is decidable by some non

deterministic Turing machine in polynomial time.
 Remember that the model we have used so far is a
deterministic Turing machine
 It is provable that a Non Deterministic Turing Machine is
equivalent to a Deterministic Turing Machine
 Remember NFA to DFA conversion?
 Given an NFA with n states how many states does the
equivalent DFA have?
 Worst case …. 2n
 The deterministic version of a poly time non deterministic
Turing machine will run in exponential time (worst case)
 Complexity Class NP
 Non-deterministic part as well
 choose(b): choose a bit in a non-deterministic way and assign to b
 If someone tells us the solution to a problem, we can verify it in
polynomial time
 Two Properties: non-deterministic method to generate possible
solutions, deterministic method to verify in polynomial time that
the solution is correct.
Example

 Here are a few examples for decision problems in NP:

Traveling salesperson:
Subset Sum
Linear Programming
Integer Programming
Graph isomorphism:
Composite numbers:
Connectivity:
Factoring
Example

 Graph theory has these fascinating(annoying?) pairs of

problems
Shortest path algorithms?
Longest path is NP complete (we’ll define NP complete
later)
Eulerian tours (visit every vertex but cover every edge
only once, even degree etc). Solvable in polynomial time!
Hamiltonian tours (visit every vertex, no vertices can be
repeated). NP complete
Co-NP Class

 Co-NP stands for the complement of NP Class. It means if the

answer to a problem in Co-NP is No, then there is proof that can
be checked in polynomial time.
Features:
 If a problem X is in NP, then its complement X’ is also in Co-
NP.
 For an NP and Co-NP problem, there is no need to verify all the
answers at once in polynomial time, there is a need to verify only
one particular answer “yes” or “no” in polynomial time for a
problem to be in NP or Co-NP.
 To check prime number
 Integer Factorization
Relation of P and NP

 P is a subset of NP
 “P = NP”?
 Language L is in NP, complement of L is in co-NP
 co-NP ≠ NP
 P ≠ co-NP
Relation of P and NP

 One of the most important open problems is the question P =

NP?
 There are thousands of problems that are in NP but appear
not to be in P.
 But no proof that they are n’t really in P.
 Worth $1 million if you can solve it:
• http://www.claymath.org/millennium-problems/p-vs-np-
problem
Complexity Classes P and NP
Con…

 The P versus NP problem is a major unsolved problem in computer

science. It asks whether every problem whose solution can be quickly
verified (technically, verified in polynomial time) can also be solved
quickly (again, in polynomial time).
What is not in NP?

 Un decidable problems
 Given a polynomial with integer coefficients, does it have
integer roots
 Hilbert’s nth problem
 Impossible to check for all the integers
 Even a non-deterministic TM has to have a finite number of
states!
 More on decidability later
 Tautology
A Boolean formula that is true for all possible assignments
Here just one ‘verifier’ will not work. You have to try all
possible values
 Polynomial-Time Reducibility
 Language L is polynomial-time reducible to language M if there
is a function computable in polynomial time that takes an input
x of L and transforms it to an input f(x) of M, such that x is a
member of L if and only if f(x) is a member of M.
 Shorthand, LpolyM means L is polynomial-time reducible to M


NP - hard

 What are the hardest problems in NP?

 That notation means that L1 is reducible in polynomial time

to L2 .
 The less than symbol basically means that the time taken to
solve L1 is no worse that a polynomial factor away from the
time taken to solve L2.
NP-hard

 A problem (a language) is said to NP-hard if every problem

in NP can be poly time reduced to it.

Examples: Halting problem, Vertex cover problem

 NP Complete problems/languages

 Need to be in NP
 Need to be in NP-Hard
 A decision problem D is NP-complete iff
 D  NP
 every problem in NP is polynomial-time reducible to D
 If both are satisfied then it is an NP complete problem
 Reducibility is a transitive relation.
 If we know a single problem in NP-Complete that helps when we are asked to
prove some other problem is NP-Complete

 Assume problem P is NP Complete

All NP problems are reducible to this problem
 Now given a different problem P’
If we show P reducible to P’
Then by transitivity all NP problems are reducible to P’
 Complexity Class NP­Complete
 In complexity theory, the NP ­complete problems are the most
difficult problems in NP ("non­deterministic polynomial time") in
the sense that they are the ones most likely not to be in P.
 If one could find a way to solve any NP­complete problem quickly
(in polynomial time), then they could use that algorithm to solve
all NP problems quickly.
 At present, all known algorithms for NP­complete problems
require time that is super polynomial in the input size.
 To solve an NP ­complete problem for any nontrivial problem size,
generally one of the following approaches is used:
Approximation
Probabilistic
Special cases
Heuristic
Some well­known problems that are NP ­complete are:

 Boolean satisfiability problem (SAT)

 N­puzzle
 Knapsack problem
 Hamiltonian cycle problem
 Traveling salesman problem
 Sub graph isomorphism problem
 Subset sum problem
 Clique problem
 Vertex cover problem
 Independent set problem
 Graph coloring problem
 NP-Hard and NP-Complete
 Restriction: A known NP-complete problem M is actually just
a special case of L
 Local replacement: reduce a known NP-complete problem M to
L by dividing instances of M and L into “basic units” then
showing each unit of M can be converted to a unit of L
 Component design: reduce a known NP-complete problem M to
L by building components for an instance of L that enforce
important structural functions for instances of M.
 NP-hard NP-Complete

NP-Hard problems(say X) can be solved if and NP-Complete problems can be solved by a non-
only if there is a NP-Complete problem(say Y) deterministic Algorithm/Turing Machine in
that can be reducible into X in polynomial time. polynomial time.

To solve this problem, it do not have to be in To solve this problem, it must be both NP and
NP . NP-hard problems.

Time is unknown in NP-Hard. Time is known as it is fixed in NP-Hard.

NP-Complete is exclusively a decision

NP-hard is not a decision problem.
problem.

Not all NP-hard problems are NP-complete. All NP-complete problems are NP-hard

Do not have to be a Decision problem. It is exclusively a Decision problem.

It is optimization problem used. It is Decision problem used.

Example: Determine whether a graph has a

Example: Halting problem, Vertex cover Hamiltonian cycle, Determine whether a
problem, etc. Boolean formula is satisfiable or not, Circuit-
Hierarchy

• P  NP  PSPACE  EXPTIME  NEXPTIME  EXPSPACE

Con…
Complexity Class Characteristic feature

Easily solvable in polynomial

P
time.

Yes, answers can be checked in

NP
polynomial time.

No, answers can be checked in

Co-NP
polynomial time.

All NP-hard problems are not in

NP-hard NP and it takes a long time to
check them.

A problem that is NP and NP-

NP-complete
hard is NP-complete.
Individual Assignment 2
1. Explain in detail and Give at least 5 problem examples for each of
the following class type listed bellow?
A. P- Class
B. NP- Class
C. NP-Hard and NP-Complete
E. L, NL, NL-Complete