Advanced Artificial

Intelligence

Simulated Annealing
Prof Salwani Abdullah
salwani@ukm.edu.my
 Simulated Annealing (SA)

• In hill-climbing (HC)
– moves are always to better states
– this gets stuck in local optima
• To escape a local optimum we must allow
worsening moves

• SA is a controlled way to allow downwards

(“wrong-way”, worsening) steps
2
 Simulated Annealing
• Motivated by the physical annealing process

• Material is heated and slowly cooled into a

uniform structure

• Simulated annealing mimics this process.

• The first SA algorithm was developed in 1953

(Metropolis)
 Simulated Annealing
• Compared to hill climbing the main difference is
that SA allows downwards steps

• Simulated annealing also differs from hill

climbing in that a move is selected at random
and then decides whether to accept it

• In SA better moves are always accepted.

Worsening moves are sometimes selected.
 Simulated Annealing
• Kirkpatrick (1983) applied SA to
optimisation problems

• Kirkpatrick, S , Gelatt, C.D., Vecchi, M.P. 1983.

Optimization by Simulated Annealing. Science,
vol 220, No. 4598, pp 671-680
 The Problem with Hill Climbing
• Gets stuck at local minima

• Possible solutions

• Try several runs, starting at different positions

• Increase the size of the neighbourhood (e.g.

in TSP try 3-opt rather than 2-opt)
 To accept or not to accept?
• The law of thermodynamics states that at
temperature, t, the probability of an
increase in energy of magnitude, δC, is
given by

P(δc) = exp(-δc /kt)

• Where k is a constant known as Boltzmann’s

constant
 To accept or not to accept - SA?
P = exp(-c/t) > r
• Where
– c is change in the evaluation function
– t the current temperature
– r is a random number between 0 and 1
 To accept or not to accept - SA?

Change, C Temp, T exp (-C/T) Change, C Temp, T exp (-C/T)

0.2 0.95 0.810157735 0.2 0.1 0.135335283

0.4 0.95 0.656355555 0.4 0.1 0.018315639
0.6 0.95 0.531751530 0.6 0.1 0.002478752
0.8 0.95 0.430802615 0.8 0.1 0.000335463
 To accept or not to accept - SA?
• The probability of accepting a worse state is
a function of both the temperature of the
system and the change in the cost function

• As the temperature decreases, the

probability of accepting worse moves
decreases

• If t=0, no worse moves are accepted (i.e. hill

climbing)
 SA Algorithm
• The most common way of implementing an
SA algorithm is to implement hill climbing
with an accept function and modify it for SA
 SA Algorithm
Generate an initial solution at random, Sol;
Calculate initial cost function value, f(Sol); Set Solbest = Sol;
Set maximum number of iterations, NumOfIte;
Set initial temperature, T0; final temperature Tf;
Cooling schedule, Delta = (T0 – Tf)/NumOfIte
Set t = T0
Set iteration = 0;
Do while (iteration <= NumOfIte);
{
Define a new solution by applying a neighbourhood structure, Sol*;
Evaluate new solution f(Sol*);
if (f(Sol*) is better than f(Solbest))
Sol = Sol*;
Solbest = Sol*;
else
c = f(Sol*)- f(Sol);
Generate RandNum, a random number in [0,1];
if (RandNum < e (-c/t) )
Sol = Sol*;
t = t – Delta;
}
 SA Algorithm - Observations
• The cooling schedule is hidden in this
algorithm - but it is important

• The algorithm assumes that annealing will

continue until temperature is zero - this is
not necessarily the case
 SA Cooling Schedule
• Starting Temperature

• Final Temperature

• Temperature Decrement

• Iterations at each temperature

 SA Cooling Schedule - Starting
Temperature
• Starting Temperature

– Must be hot enough to allow moves to almost

neighbourhood state (else we are in danger of
implementing hill climbing)
– Must not be so hot that we conduct a random
search for a period of time
– Problem is finding a suitable starting
temperature
 SA Cooling Schedule - Starting
Temperature
• Starting Temperature - Choosing

– If we know the maximum change in the cost

function we can use this to estimate
– Start high, reduce quickly until about 60% of
worse moves are accepted. Use this as the
starting temperature
– Heat rapidly until a certain percentage are
accepted the start cooling
 SA Cooling Schedule - Final
Temperature
• Final Temperature - Choosing
– It is usual to let the temperature decrease until it
reaches zero
However, this can make the algorithm run for a
lot longer, especially when a geometric cooling
schedule is being used

– In practise, it is not necessary to let the

temperature reach zero because the chances of
accepting a worse move are almost the same as
the temperature being equal to zero
 SA Cooling Schedule - Final
Temperature
• Final Temperature - Choosing

– Therefore, the stopping criteria can either be

a suitably low temperature or when the
system is “frozen” at the current temperature
(i.e. no better or worse moves are being
accepted)
 SA Cooling Schedule - Temperature Decrement

• Temperature Decrement

– Theory states that we should allow enough

iterations at each temperature so that the
system stabilises at that temperature
– Unfortunately, theory also states that the
number of iterations at each temperature to
achieve this might be exponential to the
problem size
 SA Cooling Schedule - Temperature Decrement

• Temperature Decrement

– We need to compromise
– We can either do this by doing a large number
of iterations at a few temperatures, a small
number of iterations at many temperatures or
a balance between the two
 SA Cooling Schedule - Temperature Decrement

• Temperature Decrement
– Linear
•temp = temp - x
– Geometric
•temp = temp * x
•Experience has shown that x should be between 0.8 and
0.99, with better results being found in the higher end of
the range. Of course, the higher the value of x, the
longer it will take to decrement the temperature to the
stopping criterion
 SA Cooling Schedule - Iterations

• Iterations at each temperature

– A constant number of iterations at each
temperature

– Another method, first suggested by

(Lundy, 1986) is to only do one iteration
at each temperature, but to decrease
the temperature very slowly.
 SA Cooling Schedule - Iterations

• Iterations at each temperature

– An alternative is to dynamically change the
number of iterations as the algorithm
progresses

At lower temperatures it is important that a

large number of iterations are done so that
the local optimum can be fully explored

At higher temperatures, the number of

iterations can be less
 Travelling Salesman Problem
• Travelling Salesman Problem (TSP): Given a set
of cities and distance between every pair of
cities, the problem is to find the shortest
possible route that visits every city exactly
once and returns to the starting point.
• is a minimization problem
• represented as a complete graph
 Travelling Salesman Problem
The traveling salesman problem (TSP) consists of
a salesman and a set of cities. Salesman has to
visit each one of the cities starting from a
certain one and returning to the same city. The
objective is to miminise the total distance of the
trip (a shortest path problem). The travelling
salesman problem can be presented as a full
connected graph where the node represents a
city, and the edge represents the distance
between cities.
 TSP as a complete graph

TSP is a (minimisation)NP problem

size = n = number of cities
time to solve the problem T(n) ~ 2n or n! >>> n, n2

if n = 5, so a number of possible solution (path) = 5! = 5x4x3x2x1 = 120

What if n = 1000?
 SA for TSP
Initialization:
T0 = 0.95; Tf = 0.1; MaxIter = 100;
Delta = (T0-Tf)/(MaxIter) = 0.0085;
T = T0 ;
iter = 1;
Initial solution construction phase:
//Generate an initial solution at random
//Note that a solution is represented as a
sequence of cities (path)
Sol = C-B-A-D-E-C
f(Sol) = 6+1+4+1+5= 17
Solbest = Sol = C-B-A-D-E-C
f(Solbest) = f(Sol) = 17
Improvement phase:
//in the do while loop
do while (iter <=MaxIter)
//generate a new solution using a swap
neighbourhood structure i.e., swap.
Assume we swap between cities A and C in the
Sol = C-B-A-D-E-C.

First iteration
Sol* = A-B-C-D-E-A;
f(Sol*) = 1+6+4+1+3=15
if f(Sol*) better than f(Sol) // 15 < 17 -> true
Sol = Sol* = A-B-C-D-E-A
f(Sol) = f(Sol*) = 15
if f(Sol*) better than f(Solbest)
Solbest = Sol* = A-B-C-D-E-A
f(Solbest) = f(Sol*) = 15

//Update T
T = T – Delta = 0.95-0.0085=0.9442
if exp(-c/T) > r
//assume exp (-1/0.9442) = 0.000013

0.000013 > 0.00000007 => true

then
Sol = Sol* = A-B-C-D-E-A
f(Sol) = f(Sol*) = 16
else
Sol = Sol
T=T-Delta = 0.9442-0.0085= 0.9357;
Second iteration:
Sol = A-B-C-D-E-A
Sol* = A-B-D-C-E-A //swap between C and D
f(Sol*) = 1+2+5+5+3 = 16
if f(Sol*) better than f(Sol) //16 < 15)
….
else
c = f(Sol*) – f(Sol) = 16-15 = 1
generate a random number, r [0,1]
// assume r = 0.00000007
Third iteration:
Sol = A-B-D-C-E-A
Sol* = A-C-D-B-E-A //swap between B and C
f(Sol*) = 7+4+2+5+3 = 21
if f(Sol*) better than f(Sol) //21 < 16)
….
else
c = f(Sol*) – f(Sol) = 21-16 = 5
generate a random number, r [0,1]
// assume r = 0.00023
if exp(-c/T) > r
//assume exp (-6/0.9357) = 0.00046
//0.00014 > 0.00023 => false
then
…
else
Sol = Sol //A-B-D-C-E-A

T=T-Delta=0.9357-0.0085=0.9272;
 SA for TSP

17.5

17

16.5
Distance

16

15.5

15

14.5
0 1 2 3 4 5 6 7 8 9 10
Iteration

f(Sol)
Fourth iteration:
Sol = A-B-D-C-E-A
Sol* = ???
…
//repeat the process until the maximum number
of iteration is met.
//return the best solution and the quality of the
best solution.
End of Simulated Annealing