Module 3 (6Hrs)

• Optimal Search Algorithms

– Global optimization algorithms – Genetic
Algorithms, Particle Swarm Optimization
Algorithm, Ant Colony Optimization, Gravitational
Search Algorithm – Games – Optimal Decisions in
Games – Minimax Algorithm, Alpha-Beta Pruning
Algorithm

Thursday, 01 December 2022 Instructor: Dr. Kovendan AKP 1

 Local beam search
• Keeping just one node in memory might seem to
be an extreme reaction to the problem of
memory limitations
• local beam search algorithm – k states.
• In a local beam search, useful information is
passed among the parallel search threads
• local beam search – suffer from a lack of diversity
among the k states—they can quickly become
concentrated in a small region – stochastic beam
search
Thursday, 01 December 2022 Instructor: Dr. Kovendan AKP 2
 Genetic algorithm
• Successor states are generated by combining two
parent states rather than by modifying a single
state
• Like beam searches, GAs begin with a set of k
randomly generated states – population
• Each state is rated by the objective function –
fitness function
• Two pairs are selected at random for
reproduction, in accordance with the
probabilities
Thursday, 01 December 2022 Instructor: Dr. Kovendan AKP 3
 POPULATION

FITNESS
FUNCTION

SELECTION

CROSSOVER

MUTATION

OPTIMIZED?

Thursday, 01 December 2022 Instructor: Dr. Kovendan AKP 4

 Genetic algorithm

Thursday, 01 December 2022 Instructor: Dr. Kovendan AKP 5

 Swarm Intelligence

• Composed of many individuals

• Individuals are homogeneous
• Local interaction based on simple rules
• Self-organization ( No centralized Control)
Algorithms
– Particle Swarm Optimization (PSO)
– Ant Colony Optimization
– Artificial Bee Colony Algorithm
– Artificial Immune Systems Algorithm

Thursday, 01 December 2022 Instructor: Dr. Kovendan AKP 6

 Components of Search Techniques
• Initial solution
• Search direction
• Update criteria
• Stopping criteria
• All above elements can be either
– Deterministic or Stochastic
– Single points or population based

Thursday, 01 December 2022 Instructor: Dr. Kovendan AKP 7

 Metaheuristics
• Metaheuristics are (roughly) high-level strategies that
combining lower-level techniques for exploration and
exploitation of the search space.
• Metaheuristics are strategies that “guide” the search process.
• The goal is to efficiently explore the search space in order to
find (near-)optimal solutions.
• Metaheuristic algorithms are approximate and usually non-
deterministic.
• Metaheuristics are not problem-specific.
• Metaheristcs refers to algorithms including
• -Evolutionary Algorithms
• Swarm Intelligence , Genetic Algorithm , Differential Evolution , Evolutionary programing
• - Simulated Annealing

Thursday, 01 December 2022 Instructor: Dr. Kovendan AKP 8

 Particle Swarm Optimization (PSO)

 PSO is stochastic optimization technique

proposed by Kennedy and Eberhart ( 1995).
 A population based search method with
position of particle is representing solution and
Swarm of particles as searching agent.
 PSO is a robust evolutionary optimization
technique based on the movement and
intelligence of swarms.
 PSO find the minimum value for the function.
Thursday, 01 December 2022 Instructor: Dr. Kovendan AKP 9
Particle Swarm Optimization (PSO)
• The basic concept of PSO lies in accelerating each
particle toward its pbest and the gbest locations,
with a random weighted acceleration at each time.
• The idea is similar to bird flocks searching for
food.
– Bird = a particle, Food = a solution
– pbest = the best solution (fitness) a particle has achieved
so far.
– gbest = the global best solution of all particles within
the swarm

Thursday, 01 December 2022 Instructor: Dr. Kovendan AKP 10

 Global best

New Velocity

Position X Personal best

Thursday, 01 December 2022 Instructor: Dr. Kovendan AKP 11

 Particle Swarm Optimization (PSO)
Each particle tries to modify its position X using the following
formula:

X (t+1) = X(t) + V(t+1) (1)

V(t+1) = wV(t) + c1 ×rand ( ) × ( Xpbest - X(t)) + c2 ×rand ( ) × ( Xgbest - X(t)) (2)

V(t) velocity of the particle at time t

X(t) Particle position at time t
w Inertia weight
c1 , c2 learning factor or accelerating factor
rand uniformly distributed random number
between 0 and 1
Xpbest particle’s best position
Xgbest global best position

Thursday, 01 December 2022 Instructor: Dr. Kovendan AKP 12

 PSO Algorithm
The PSO algorithm pseudocode:
Input: Randomly initialized position and velocity of Particles:
Xi (0) andVi (0)
Output: Position of the approximate global minimum X*

1: while terminating condition is not reached do

2: for i = 1 to number of particles do
3: Calculate the fitness function f
4: Update personal best and global best of each particle
5: Update velocity of the particle using Equation 2
6: Update the position of the particle using equation 1
7: end for
8: end while
Thursday, 01 December 2022 Instructor: Dr. Kovendan AKP 13
 PSO Algorithm

Thursday, 01 December 2022 Instructor: Dr. Kovendan AKP 14

 Ant Colony Optimization
“Ant Colony Optimization (ACO) studies artificial systems
that take inspiration from the behavior of real ant
colonies and which are used to solve discrete
optimization problems.”

Thursday, 01 December 2022 Instructor: Dr. Kovendan AKP 15

 Ant Colony Optimization
 Probalistic Techniques to solve optimization Problem
 It is a population based metaheuristic used to find approximate solution to
an optimization problem.
 The Optimization Problem must be written in the form of path finding with
a weighted graph

Application of ACO
 Shortest paths and routing
 Assignment problem
 Set Problem

Thursday, 01 December 2022 Instructor: Dr. Kovendan AKP 16

 Idea
• The way ants find their food in shortest path is
interesting.
• Ants hide pheromones to remember their path.
• These pheromones evaporate with time.
• Whenever an ant finds food , it marks its return
journey with pheromones.
• Pheromones evaporate faster on longer paths.
• Shorter paths serve as the way to food for most of the
other ants.
• The shorter path will be reinforced by the pheromones
further.
• Finally , the ants arrive at the shortest path.

Thursday, 01 December 2022 Instructor: Dr. Kovendan AKP 17

 ACO Concept
• Ants navigate from nest to food source. Ants are blind!
• Shortest path is discovered via pheromone trails. Each ant
moves at random
• Pheromone is deposited on path
• More pheromone on path increases probability of path
being followed
• Algorithm
– Construct AntSolutions: Partial solution extended by adding an
edge based on stochastic and pheromone considerations.
– Apply LocalSearch: problem-specific, used in state-of-art ACO
algorithms.
– Update Pheromones: increase pheromone of good solutions,
decrease that of bad solutions (pheromone evaporation).

Thursday, 01 December 2022 Instructor: Dr. Kovendan AKP 18

 Ant Colony Algorithm

Thursday, 01 December 2022 Instructor: Dr. Kovendan AKP 19

 Summary
• Swarm Intelligence is population based search technique.

• PSO is robust stochastic optimization technique and can be applied

for data clustering.

• In ACO algorithm, the optimization problem must be written in the

form of path finding with a weighted graph

• PSO clustering algorithm avoid the problems that arise with Kmeans
clustering such as terminating at local minimum, generating empty
clusters and sensitivity to noisy data and outliers.

Thursday, 01 December 2022 Instructor: Dr. Kovendan AKP 20

 GSA
• The GSA could be considered as an isolated system of objects (particles)
with masses. In this system, objects obey the following laws:

Law of Gravity: each particle attracts every other particle and the
gravitational force between two particles is directly proportional to the
product of their masses and inversely proportional to power p of the distance
between them, R^p.
Law of Motion: the current velocity of any particle is equal to the sum of the
fraction of its previous velocity and the variation in the velocity. Variation in
the velocity or acceleration of any particle is equal to the force acted on the
system divided by its mass.

Thursday, 01 December 2022 Instructor: Dr. Kovendan AKP 21

Gravitational Search Algorithm (GSA)

objects

• Position
solution of the problem

• The mass value

fitness function

The gravitational force is an information-transferring tool.

A heavy mass has a large effective attraction radius.
Thursday, 01 December 2022 Instructor: Dr. Kovendan AKP 22
Gravitational Search Algorithm (GSA)

Law of gravity
N
2
Law of motion 1 2

Thursday, 01 December 2022 Instructor: Dr. Kovendan AKP 23

 STARTt

GSA Parameter setting

Generate initial population

Evaluate each agent

Update G, best, and worst of the

population

Calculate mass (M) , forces (F), and

acceleration (a) for each agent

Update velocity (V), and position (X)

Meeting end of
Criterion?

Thursday, 01 December 2022 Return best

Instructor: solution
Dr. Kovendan AKP 24
GSA

t=1 t=2 t=T

.Evaluation .Evaluation
.Evaluation .Mass calculation .Mass calculation
.Mass calculation .F, a, V .F, a, V
.Calculation of F, a, V .X=X+V .X=X+V
.X=X+V

t : time / iteration
Thursday, 01 December 2022 Instructor: Dr. Kovendan AKP 25
 Initialization
• At the first iteration, each agents is randomly
allocated in the search space.
• xd,hi and xd,low are the boundaries of the variable d.
x id (t  1) ~ U(x d, low , x d, hi )
 
xid  xid ,lo  xid , hi  xid ,lo  xin for d  1, 2, ..., m
i  1,2,..., N

Thursday, 01 December 2022 Instructor: Dr. Kovendan AKP 26

 Evaluation
• At the start of each iteration, agents are evaluated.
• For each agent, the problem is solved with the variable values
suggested by that agent and the objective function is
calculated.
• To this, most of the time, the problem is simulated using a
software.
Problem
𝑋1 𝐹𝑖𝑡1
𝑋2 𝐹𝑖𝑡2
. .
𝑋= . .
. .
𝑋𝑁 𝐹𝑖𝑡𝑁

Total Number of fitness evaluations (FEs)

Thursday, 01 December 2022 Instructor: Dr. Kovendan AKP 27

 Mass calculation

• fiti (t) is the fitness of agent i at time t.

i=1,…,N fiti (t )  worst (t )

qi (t ) 
best (t )  worst (t )

qi (t )
M i (t ) 
N
 q j (t )
j 1

M (t )  M (t )  M (t )  M (t )
ai pi ii i
Thursday, 01 December 2022 Instructor: Dr. Kovendan AKP 28
 Mass calculation
• In minimization problems:

• In maximization problems:

Thursday, 01 December 2022 Instructor: Dr. Kovendan AKP 29

 Gravitational constant
• The gravitational constant is a function of time that takes an
initial value of G0 and reduced during iterations.

G (t )  G (G0 , t )

• An example function for G :

t

G (t )  G0 e T

Thursday, 01 December 2022 Instructor: Dr. Kovendan AKP 30

 Force calculation
• The force applied to agent i from agent j at dimension d :

M (t )  M (t )
d aj pi d (t )  x d (t ) )
F (t )  G (t ) ( x
ij R (t ) P   j i
ij
d  1, 2,  , m and i  1, 2,  , N

• R is the distance between object i and j :

Thursday, 01 December 2022 Instructor: Dr. Kovendan AKP 31

 Force calculation
• Total force applied to object i at dimension d :

i=1,…,N d=1,…,m

• Kbest is the set of ‘k’ heaviest objects.

• In each iteration, only ‘k’ heaviest objects apply force to
the others.
• ‘k’ is initialized to ‘N’ and decreased with time.
Thursday, 01 December 2022 Instructor: Dr. Kovendan AKP 32
 Force
• m=2
• k=2
• N=10

Thursday, 01 December 2022 Instructor: Dr. Kovendan AKP 33

 Acceleration and velocity calculation
• Acceleration of agent i at dimension d :

• Velocity of agent i at dimension d :

d d d
vi (t  1)  randi  vi (t )  ai (t )

Thursday, 01 December 2022 Instructor: Dr. Kovendan AKP 34

 Position updating

• Next position of agent i at dimension d:

d d d
xi (t  1)  xi (t )  vi (t  1)

Thursday, 01 December 2022 Instructor: Dr. Kovendan AKP 35

 ADVERSARIAL SEARCH

• Rich tradition of creating game-playing programs in AI

• Many similarities to search
• Most of the games studied
– have two players,
– are zero-sum: what one player wins, the other loses
– have perfect information: the entire state of the game is known to both
players at all times
• E.g., tic-tac-toe, checkers, chess, Go, backgammon, …
• Will focus on these for now
• Recently more interest in other games
– Esp. games without perfect information; e.g., poker
• Need probability theory, game theory for such games

Thursday, 01 December 2022 Instructor: Dr. Kovendan AKP 36

 Games
• A game can be formally defined as a kind of
search problem with the following elements:
– So
– PLAYER(s)
– ACTIONS(s)
– RESULT(s, a)
– TERMINAL-TEST(s)
– UTILITY(s, p)

Thursday, 01 December 2022 Instructor: Dr. Kovendan AKP 37

 “Sum to 2” game
• Player 1 moves, then player 2, finally player 1 again
• Move = 0 or 1
• Player 1 wins if and only if all moves together sum to 2
Player 1
0 1

Player 2 Player 2

0 1 1
0

Player 1 Player 1 Player 1 Player 1

0 1 0 1 0 1 0 1

-1 -1 -1 1 -1 1 1 -1
Player 1’s
Thursday, 01 utility
Decemberis2022
in the leaves; player 2’s utility
Instructor: is the
Dr. Kovendan AKPnegative of this 38
 Backward induction (aka. minimax)
• From leaves upward, analyze best decision for player at node, give
node a value
– Once we know values, easy to find optimal action (choose best value)

1 Player 1
0 1

Player 2 Player 2
-1 1
0 1 1
0

Player 1 -1 Player 1 1 1 Player 1 1 Player 1

0 1 0 1 0 1 0 1

-1 -1 -1 1 -1 1 1 -1
Thursday, 01 December 2022 Instructor: Dr. Kovendan AKP 39
 Modified game
• From leaves upward, analyze best decision for
player at node, give node a value

6 Player 1
0 1

Player 2 Player 2
-1 6
0 1 1
0

Player 1 -1 Player 1 4 6 Player 1 7 Player 1

0 1 0 1 0 1 0 1

-1 -2 -3 4 -5 6 7 -8
Thursday, 01 December 2022 Instructor: Dr. Kovendan AKP 40
 A recursive implementation
• Value(state)
• If state is terminal, return its value
• If (player(state) = player 1)
– v := -infinity
– For each action
• v := max(v, Value(successor(state, action)))
– Return v
• Else
– v := infinity Space? Time?
– For each action
• v := min(v, Value(successor(state, action)))
– Return v
Thursday, 01 December 2022 Instructor: Dr. Kovendan AKP 41
Thursday, 01 December 2022 Instructor: Dr. Kovendan AKP 42