Ant Colony Optimization for solving a Combinatorial Optimization Problem

(Understanding by an example)

Ant Colony Optimization (ACO) is swarm intelligence based optimization algorithm developed
by Initially proposed by an Italian researcher Marco Dorigo in 1992 in his PhD thesis [1].
Originally the ACO was developed to find an optimal path in a connected graph which is a
discrete optimization problem. Most of the time ACO has been applied to discrete
optimization problems and it also performs very well on this class of optimization problems.

The algorithm is inspired by foraging behavior of ants of a colony, therefore the name Ant
Colony Optimization. During the search for food source, ants behave intelligently and thus
always find the shortest path from their nest (home) to the food source.

Source: Google Images

In this example, we have considered a discrete optimization problem. The objective of this
example is to explain the working procedure of ACO in simplistic way. Understanding this
example will help readers to code ACO, to apply ACO for a real world problem or to modify
the ACO.

The example will be explained in 4 steps:

1. Problem Definition
2. ACO Parameters
3. Initialization
4. Main Loop

1. Problem Definition

𝑀𝑖𝑛 𝑓 (𝑥 ) = 𝑥*+ + 𝑥++ ,

Search space: 𝑥* ∈ {2, 3.2, 6.5, 7, 9}, 𝑥+ ∈ {1.2, 3, 5.9, 6}

 It is clear that 𝑥* and 𝑥+ can attain values from a finite set and therefore it is a discrete
optimization problem. We need to find out optimal value of 𝑥 = (𝑥* , 𝑥+ ) which
minimizes function 𝑓(𝑥).

2. ACO Parameters

Dimension of the problem = 2

Swarm size (number of ants) =5
Evaporation rate 𝜌 = 0.5

3. Initialization

Initially each ant can choose any path (5 choices/possible paths for 𝑥* and 4 choices/possible
paths for 𝑥+ ). Ants choose any path based on the pheromone amount 𝜏<= . Here 𝑖 is for variable
number (𝑖 = 1, 2) and j is the corresponding number of possible values (if 𝑖 = 1, 𝑗 = 1, 2, … ,5
and if 𝑖 = 2, 𝑗 = 1,2, … ,4 ). Clearly, total possible solutions are 5 × 4 = 20 out of which we
need to find one optimal solution.

𝜏<= = amount of pheromone on the path if 𝑖 BC variable attains 𝑗BC value.

Initially, we consider same amount of pheromone on each path. Let 𝜏<= = 1 ∀ 𝑖 =
1, 2 and 𝑗 = 1, 2,…,5/4. Equal amount of pheromone implies the equal probabilities of
selecting a path.
 Fig. 1: Representation of all possible solutions or paths

4. ACO Position Update

Iteration 1:

Now for each ant, the probability of selecting path 𝑥<= (choosing a value of variables) is
calculated using the following formulae:

𝜏*= 1
𝑝*= = G = , ∀ 𝑗 = 1,2, … ,5
∑BH* 𝜏*B 5
and

𝜏+= 1
𝑝+= = I = , ∀ 𝑗 = 1,2, … ,4
∑BH* 𝜏+B 4

Now we will apply the Roulette Wheel Selection to choose a specific path. For this, we
calculate cumulative probability ranges associated with each path given below:
 Cumulative Probability Ranges for 𝒙𝟏 Cumulative Probability Ranges for 𝒙𝟐
1 1
M0, N = (0, 0.2) M0, N = (0,0.25)
5 4
1 2 1 2
M , N = (0.2, 0.4) M , N = (0.25, 0.5)
5 5 4 4
2 3 2 3
M , N = (0.4, 0.6) M , N = (0.5, 0.75)
5 5 4 4
3 4 3 4
M , N = (0.6, 0.8) M , N = (0.75, 1)
5 5 4 4
4
M , 1N = (0.8, 1)
5

In order to choose specific path using Roulette Wheel Selection, we generate a uniform
random number in (0, 1) for each ant and for each variable. Here 5 × 2 = 10 random numbers
will be generated as the number of ants are 5 and number of variables are 2.

Let

𝑟** =0.1622 𝑟+* =0.6020

𝑟*+ =0.7943 𝑟++ =0.2630
𝑟*Q =0.3112 𝑟+Q =0.6541
𝑟*I =0.5285 𝑟+I =0.6892
𝑟*G =0.1656 𝑟+G =0.1482

Ants movement in direction 𝒙𝟏

Now since 𝑟** ∈ (0, 0.2), the first ant will choose the first path, i.e. 𝑥** = 2. In other
words, first solution’s first variable value is 𝑥** = 2.

Again, since 𝑟*+ ∈ (0.6, 0.8), the second ant will choose 4th path or the second
solution’s first variable is 𝑥*I = 7.

Similarly, 𝑟*Q ∈ (0.2, 0.4) implies third solution’s first variable is 𝑥*+ = 3.2

𝑟*I ∈ (0.4, 0.6) implies fourth solution’s first variable is 𝑥*Q = 6.5

𝑟*G ∈ (0, 0.2) implies fifth solution’s first variable is 𝑥** = 2

Similarly, we will calculate the paths for variable 𝑥+

Ants movement in direction 𝒙𝟐

𝑟+* ∈ (0.5, 0.75) implies first solution’s second variable is 𝑥+Q = 5.9

𝑟++ ∈ (0.25, 0. 5) implies second solution’s second variable is 𝑥++ = 3

 𝑟+Q ∈ (0.5, 0.75) implies third solution’s second variable is 𝑥+Q = 5.9

𝑟+I ∈ (0.5, 0.75) implies fourth solution’s second variable is 𝑥+Q = 5.9

𝑟+G ∈ (0, 0.25) implies fifth solution’s second variable is 𝑥+* = 1.2

Thus the five solutions (five ants) are (let 𝑘 BC solution is denoted by 𝑋T ):

𝑋* = (2, 5.9)
𝑋+ = (7, 3)
𝑋Q = (3.2, 5.9)
𝑋I = (6.5, 5.9)
𝑋G = (2, 1.2)

Now we will calculate the objective function values corresponding to each solution (Let
objective function value corresponding to 𝑘 BC solution is denoted by 𝑓T ):

𝑓* = 38.8100
𝑓+ = 58.0000
𝑓Q = 45.0500
𝑓I = 77.0600
𝑓G = 5.4400

It can be seen that the 5th ant took the best path (2, 1.2) with objective function value 5.4400
while the 4th ant took the worst path (6.5, 5.9) with objective function value 77.0600. So after
first iteration

𝑓UVWB = 5.4400
𝑓XYZWB = 77.0600

Fig. 2: After iteration 1

At this stage, we check if all the ants follow a single path. If all the ants follow a single path
then the algorithm is said to converge and we declare that path (solution) as the optimal path
(solution). It can be seen that all the paths are not same and therefore the algorithm has yet
not been converged and we proceed to another iteration.

Iteration 2

For iteration 2, we consider that all the ants return at home and start again for the destination
or the food source.

In the beginning of iteration 1, we assumed equal pheromone to all the paths as there was
no previous experience or knowledge of the solution paths was available. But now for
iteration 2, we have the knowledge of fitness of each path. Therefore, first we will update the
pheromone on each path. The pheromone update formula assures that the pheromone
associated with best solution(s) increases while for other solutions it decreases (evaporates).
Following pheromone update formula is used:

[VX Y]^ T
𝜏<= = (1 − 𝜌)𝜏<= + ∑T Δ𝜏<= (1)

In eq. (1), 𝜌 ∈ (0, 1] is a user defined parameter known as evaporation rate. We take 𝜌 = 0.5.
Y]^ T
𝜏<= is the pheromone amount in previous iteration when 𝑖 BC variable attains 𝑗BC value. Δ𝜏<=
is the pheromone amount laid by 𝑘 BC ant and is given by the following formula:

T b
Δ𝜏<= = 𝑄 b cdef (2)
ghief

In eq. (2), 𝑄 is a constant parameter. Usually, 𝑄 = 2. It is clear that in initial iterations when
b b
the difference between 𝑓UVWB and 𝑓XYZWB is large the ratio b cdef is small and so 𝑄 b cdef . As the
ghief ghief
b
iterations progress, the difference between 𝑓UVWB and 𝑓XYZWB will be small and the ratio b cdef
ghief
bcdef
will tend to 1 and so 𝑄 b will tend to 𝑄 = 2. This property of pheromone deposition
ghief
assures that in early iterations less pheromone is deposited to avoid the stagnation at the
suboptimal solution.

T
In equation (1), only best ant(s)/solution(s) can deposit the pheromone Δ𝜏<= . If there are
more than one best ants at any iteration then the summation extends over all those best ants.
Here in iteration 2, we have only one best ant and therefore the summation has only one
term. It should be noted that the evaporation rate 𝜌 = 0 for the best ant(s).

Finally, the pheromone update formula for best ant(s) is:

[VX Y]^ T
𝜏<= = 𝜏<= + ∑T Δ𝜏<= (3)

For other ants it is:

[VX Y]^
𝜏<= = (1 − 𝜌)𝜏<= (4)
Thus,

𝜏*= = (1 − 0.5) × 1 = 0.5 ∀ 𝑗 = 2, 3, 4, 5

𝜏+= = (1 − 0.5) × 1 = 0.5 ∀ 𝑗 = 2, 3, 4

5.4400
𝜏** = (1 − 0) × 1 + 2 = 1.1412
77.0600

5.4400
𝜏+* = (1 − 0) × 1 + 2 = 1.1412
77.0600

Now as in previous iteration, we calculate the probabilities

𝜏*= 0.5
𝑝*= = = = 0.1592 , ∀ 𝑗 = 2, … ,5
∑GBH* 𝜏*B 1.1412 + 4 × 0.5

𝜏** 1.1412
𝑝** = G = = 0.3633
∑BH* 𝜏*B 1.1412 + 4 × 0.5

𝜏*= 0.5
𝑝+= = G = = 0.1893 , ∀ 𝑗 = 2, 3, 4
∑BH* 𝜏*B 1.1412 + 3 × 0.5

𝜏** 1.1412
𝑝+* = = = 0.4321
∑GBH* 𝜏*B 1.1412 + 3 × 0.5

cumulative probability ranges associated with each path given below:

Cumulative Probability Ranges for 𝒙𝟏 Cumulative Probability Ranges for 𝒙𝟐

(0, 0.3633) (0, 0.4321)
(0.3633, 0.5225) (0.4321, 0.6214)
(0.5225, 0.6817) (0.6214, 0.8107)
(0.6817, 0.8409) (0.8107, 1)
(0.8409, 1)

From these cumulative probability calculations, it is clear that the range corresponding to the
best ant/solution is largest. Thus in Roulette Wheel Selection, more ants will choose this path.

To apply Roulette Wheel Selection, we generate 10 random numbers in the range (0, 1).

𝑟** =0.4505 𝑟*I =0.3133

𝑟*+ =0.0838 𝑟*G =0.1524
𝑟*Q =0.2290 𝑟+* =0.8258
𝑟++ =0.5383 𝑟+I =0.0782
𝑟+Q =0.9961 𝑟+G =0.4227

Following the same procedure as in iteration 1, we found the following updated solutions:
( 𝑥* ∈ {2, 3.2, 6.5, 7, 9}, 𝑥+ ∈ {1.2, 3, 5.9, 6} )

𝑋* = (3.2, 6) 𝑓* = 46.2400
𝑋+ = (2, 3) 𝑓+ = 13.0000
𝑋Q = (2, 5.9) 𝑓Q = 38.8100
𝑋I = (2, 1.2) 𝑓I = 5.4400
𝑋G = (2, 1.2) 𝑓G = 5.4400

It can be seen that now two ants are following the same best path. So after second iteration

𝑓UVWB = 5.4400
𝑓XYZWB = 46.2400

Obviously, the difference between 𝑓UVWB and 𝑓XYZWB is now small as compare to the
difference after first iteration.

Fig. 3: After iteration 2

Now we check whether the algorithm has been converged? Since all the ants are still not
following the same path therefore the algorithm is not converged and we will move to
iteration 3.
We will continue the same process till all the ants converged to the same path.