Combinatorial

Optimization
Classical optimization models

29
Combinatorial Optimization

▪ Combinatorial Optimization is a branch of Mathematical Optimization with a vast number of

applications.

▪ This class of problems is characterized by discrete decision variables and a finite search
space.

▪ The popularity of combinatorial optimization problems stems from the fact that in many real-
world problems, the objective function and constraints are of different nature (nonlinear,
nonanalytic, black box, etc.) whereas the search space is finite.

▪ It consists in finding a "best" choice among a finite (but usually very large) set of possibilities

▪ Typically, it is impractical to apply an exhaustive search as the number of possible solutions

grows rapidly with the "size" of the input to the problem.

▪ The aim of combinatorial optimization is to find more clever methods (i.e. algorithms) for 30
exploring the solution space.
Combinatorial Optimization

▪ Combinatorial problems are characterized by an input, i.e., a general

description of conditions and parameters and a question (or task, or
objective) defining the properties of a solution.

▪ They involve finding a grouping, ordering, or assignment of a discrete,

finite set of objects that satisfies given conditions.

▪ Candidate solutions are combinations of objects or solution components

that need not satisfy all given conditions.

▪ Solutions are candidate solutions that satisfy all given conditions

31
Combinatorial Optimization

Two of the arguably most prominent questions in Combinatorial Optimization

are:

▪ How quickly can one find a single (or all) optimal solution(s) of a
given problem?

▪ When dealing with a problem where, due to complexity-theoretic

reasons, it is unlikely that an optimal solution can be found
efficiently: What is the best solution quality that an efficient
algorithm can guarantee?

32
Combinatorial Optimization Problems

Formally, a Combinatorial Optimization Problem (COP) can be defined as a triple (S,f,Ω),

where:
▪ S is a given search space
▪ f is the objective function, which should be either maximized or minimized
▪ Ω is the set of constraints that have to be fulfilled to obtain feasible solutions
The goal is to find a globally optimal solution, which is in the case of a maximization problem
a solution s∗ with the highest objective value that fulfills all constraints

33
Combinatorial Optimization Problems

COPs may arise in a wide variety of important fields such as:

▪ Transportation

▪ Computer networking

▪ Telecommunications

▪ Distribution problems

34
1. The Traveling Salesman Problem

▪ The traveling salesman problem consists of a salesman and a set

of cities.

▪ The salesman has to visit each one of the cities starting from a
certain one (e.g. the hometown) and returning to the same city.

▪ The challenge of the problem is that the traveling salesman

wants to minimize the total length of the trip.

35
1. The Traveling Salesman Problem

▪ TSP can be modelled as an undirected vertex

weighted graph, such that cities are the
graph's vertices, paths are the graph's edges,
and a path's distance is the edge's weight.

▪ It is a minimization problem starting and edge

finishing at a specified vertex after having
visited each other vertex exactly once.

▪ Often, the model is a complete graph (i.e.

each pair of vertices is connected by an
edge).
36
1. The Traveling Salesman Problem
Formulation
In the directed graph D = (N ; A), with arc-costs cij , we seek the tour (a directed cycle that
contains all n cities) of minimal length.

Where:

37
2. Job Shop Scheduling Problem

The job shop scheduling problem (JSP) consists of:

▪ A finite set of n jobs
▪ Each job consists of a chain of operations
▪ A finite set of m machines
▪ Each machine can handle at most one operation at a time
▪ Each operation needs to be processed during an uninterrupted period of
a given length on a given machine
▪ Purpose is to find a schedule, that is, an allocation of the operations to
time intervals to machines, that has minimal length

38
2. Job Shop Scheduling Problem
Formulation of the JSPTWT

Total weighted tardiness

▪ Const. 1: The technological
sequence given for every job.
▪ Const. 2: The disjunctive
constraints: capture the
sequencing problem on every
machine.
▪ Consts. 3 &4: Measure the
resulting tardiness of each job.
▪ Const. 5:A job cannot start
before its release time.
Where:
▪ tj = max{0, cj −dj} : the resulting tardiness of job j in a schedule, and cj its completion time
▪ sij represents the start time for operation oij
▪ pij :the processing of the i-th operation of job j takes pij > 0 time units.
▪ dj: due date which should, but does not necessarily have to, be fulfilled in a schedule
▪ rj :release time rj ≥ 0 so that the first operation cannot start before rj
39
Complexity Aspects

▪ The first step in studying a combinatorial optimization problem is to find out whether
the problem is "easy" or "hard".
▪ Computer scientists talk about the “complexity” or “hardness” of a problem, in terms of
the speed of the fastest possible algorithm to solve the problem.
▪ For example, it’s possible to show that finding the maximum of numbers
requires n−1 comparisons.
▪ It is considered that a problem “easy” if there’s a polynomial algorithm to solve the
problem, and “hard” otherwise.
▪ Two important classes of problems:
➢ P
➢ NP 40
Complexity Aspects
Complexity Time Solvable (P)

▪ A problem in the set P can be solved by an algorithm whose

solution time grows as a polynomial function of the size of the
problem.
▪ The P set contains problems that are solvable in polynomial-time
▪ the algorithm has running time O (nk) for some constant k.
▪ Polynomial times: O (n2), O (n3), O (1), O (n log n)
▪ Not polynomial times: O (2n), O (nn), O (n!)
▪ e.g. Testing if a number is prime runs in O (n) time

41
Complexity Aspects
Complexity Time Solvable (P)

▪ P problems are called tractable because of their polynomial

running time.
▪ Examples:
➢ Shortest path problem
➢ Assignment problems
➢ Minimum spanning tree problem
➢ Linear programming problems

42
Complexity Aspects
Non-deterministic Problems (NP)

▪ NP is the class of problems that can be checked by a polynomial

running time algorithm but can (or might) not be solved in
polynomial time
▪ Most people believe that the NP problems are not polynomial
time solvable (written as P ≠ NP)
▪ Algorithms to solve NP problems require exponential time, or
greater
▪ They are called intractable

43
Complexity Aspects
Non-deterministic Problems (NP)

Traveling Salseman Problem

▪ With n cities, find a circuit which minimize the total distance ; in Symmetric TSP
dist(x,y)=dist(y,x)
▪ |S| = n!/2n = (n-1)! / 2 ;
▪ 10 cities = 181 000 solutions
▪ 20 cities = 10 000 000 000 000 000 solutions
▪ 50 cities = 100 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000 000
000 000 000 000 solutions
▪ There is 1 000 000 000 000 000 000 000 litres of water in the planet

44
Complexity Aspects
Non-deterministic Problems (NP)

▪ Examples:
➢ Traveling salesman problem
➢ Vehicle routing problem
➢ Number partitioning problem
➢ Graph coloring problem
➢ Set cover problem
➢ Scheduling problems
➢ Packing problems

45
Convexity

▪ A set S is convex if, from everywhere in S, all other points of S

are “visible.”

46
Convexity

▪ Examples of a non-convex sets:

47
Why Convexity Matters?

48
Why Convexity Matters?

▪ With a convex objective and a convex feasible region, there can be only
one optimal solution, which is globally optimal. Several methods will
either find the globally optimal solution, or prove that there is no feasible
solution to the problem. Convex problems can be solved efficiently up to
very large size.

▪ A non-convex optimization problem may have multiple feasible regions

and multiple locally optimal points within each region. It can take
time exponential in the number of variables and constraints to determine
that a non-convex problem is infeasible, that the objective function is
unbounded, or that an optimal solution is the "global optimum" across all
feasible regions.

49
Local vs Global optimum

▪ Local optimum: a solution to a problem that is better than

all other solutions that are slightly different (at nearby
points), but worse than the global optimum.

▪ Global optimum: the best possible solution to a problem

Minimization problem Maximization problem

50