Engineering Optimization: Lecture

Ain Shams University (ASU)
Faculty of Engineering
Mechatronics Department
Engineering Optimization
MCT-434
Lecture (05)
Particle Swarm Optimization (PSO)
Dr. Eng. Omar M. Shehata

Assistant Professor
Mechatronics Engineering department,
Faculty of Engineering , Ain Shams University (ASU)
Lecture (05): Particle Swarm Optimization Engineering Optimization (MCT-434s) – Spring ‘23 -- Dr. Eng. Omar M. Shehata © 1
Outlines
• Swarm Intelligence
• Particle Swarm Optimization (PSO)
• PSO Example
• Next time …
Swarm Intelligence

What is Optimization
• Finding an alternative with the most cost effective or highest achievable
performance under the given constraints, by maximizing desired factors and
minimizing undesired ones.
Swarm Intelligence
 
Optimization Problem Formulation

• The optimization solution that we search for can be formulated as:
Optimization
Solution
Permutation Binary Arithmetic

Problem Problem Problem
A C D B E 1 0 0 1 1 31 23 19 84 21
Swarm Intelligence
 
Solution Space
Maximum
• What is a Solution-Space? x y value of
All possible values for the decision variables. function (Z)
Value = 8
Value = 2
Value = 0
Solution:
Value = 0
-3
3
1
-1 -3
-1
1
Swarm Intelligence
 
Population based Optimization

Maximum
• Who searches in the Solution Space??? x y value of
function (Z)
Value = 8
POPULATION !!!
Value = 0
Value = 2 Value = 2
Value = 0
Value = 0
Value = 1.5 Solution:
3
0
-1 -3
-2
-1
Value = -0.5
Value = -2 -1
3
-2 1
3
0
Value
Value == 00 -3
-2
-1 -3
0
1
Value = -6
-2
-3 3
2
-3
Swarm Intelligence

Introduction
• Particle Swarm Optimization (PSO) is a method for solving optimization problems
based on PARTICLES that move around in the search-space based on
mathematical formulae over the particle's position and velocity.
Searching for Food
Particle Swarm Opt. (PSO)

PSO Drivers
• But how does particles fly in the search space ??
Each Particle Speed is
affected by
PSO
Drivers
Personal Global
Inertia
Best Best
Inertia
(i)
(i -1) 𝑮𝒍𝒐𝒃𝒂𝒍𝒃𝒆𝒔𝒕
𝑿𝑶𝒎𝒂𝒓
𝑿𝑶𝒎𝒂𝒓_𝒃𝒆𝒔𝒕
New
𝑷𝒆𝒓𝒔𝒐𝒏𝒂𝒍𝒃𝒆𝒔𝒕 𝑿𝑱𝒐𝒆_𝒃𝒆𝒔𝒕
Velocity (i +1)
 
PSO Stopping Criteria

• When should we stop ??
PSO Stopping
Criteria
Solution Number of
No change
Converge Iterations
• All particles converge to • Each set of particles • The allowed number

one solution (Mostly converged to a certain of Iterations is
the global maxima) maxima. reached.
 
PSO Particle
• Previously we mentioned that optimization solution can be formulated as:
Optimization
Solution
Permutation Binary Arithmetic

Problem Problem Problem
A C D B E 1 0 0 1 1 31 23 19 84 21
Particle
Fitness Function
Research Problem So we calculate the fitness of

Needs to be hybridized with each particle in the swarm
another algorithm
 
PSO Operations: (1/2)

• We are applying these equations to
For each particle in the Swarm, we the arithmetic solution particles.
conduct these 2 steps.
Particle @(i)
PSO Operations
(2 steps) 31 23 19 84 21
𝑤1 = 0.2, 𝑉𝑖 = 0
𝑐1 = 0.5, 𝑟1 = 0.235
Update Update 𝑐2 = 1.2, 𝑟2 = 0.189
Velocity Position
P_best 35 28 15 87 56
𝑑
𝑉𝑖+1 = 𝑤1 ∗ 𝑉𝑖 + 𝑐1 𝑟1 ∗ 𝑃𝑏𝑒𝑠𝑡,𝑖 − 𝑋𝑖𝑑 + 𝑐2 𝑟2 ∗ 𝐺𝑏𝑒𝑠𝑡,𝑖
𝑑
− 𝑋𝑖𝑑 G_best 25 32 10 72 98
Inertia of particle Personal Best effect term Global Best effect term (𝟎. 𝟐) 𝟎 + (𝟎. 𝟓) 𝟎. 𝟐𝟑𝟓 𝟑𝟓 − 𝟑𝟏
term + (𝟏. 𝟐)(𝟎. 𝟏𝟖𝟗)(𝟐𝟓 − 𝟑𝟏)
𝑐1 is the acceleration 𝑐2 is the acceleration
Velocity @(i+1)
𝑤1 is the weight of coefficient coefficient
this term (user -5.3
defined) 𝑟1 is randomly generated 𝑟2 is randomly
number generated number And REPEAT…

PSO Operations: (2/2)

For each particle in the Swarm, we
• We are applying these equations to
conduct these 2 steps. the arithmetic solution particles.
Particle @(i)
PSO Operations
(2 steps) 31 23 19 84 21
Velocity @(i+1)
Update Update -5.3
Velocity Position
𝑑
𝑋𝑖+1 − 𝑋𝑖𝑑 1
𝑋𝑖+1 = 𝟑𝟏 + (−𝟓. 𝟑)
𝑉𝑖+1 =
∆𝑇
Given that ∆𝑇 is equal to 1 iteration (∆𝑇 = 1)
Thus,
𝑑
𝑋𝑖+1 = 𝑋𝑖𝑑 + 𝑉𝑖+1 NEW Particle @(i+1)
25.7
New Velocity of New Position of
Sampling Time
particle particle
And REPEAT…
PSO Example

PID Tuning Example

• PID Controller tuning is one of the most important problems and using Meta-
heuristics we can find the optimal combination of gains that satisfy a certain
objective function.
Decision Variables
Control loop (PID) System response
PSO Example
 
PID Tuning Example

• PID Controller tuning is one of the most important problems, and using Meta-
heuristics we can find the optimal combination of gains that satisfy a certain
objective function.
0 ≤ 𝐾𝑝 ≤ 100
0 ≤ 𝐾𝑖 ≤ 10
0 ≤ 𝐾𝑑 ≤ 0.5
Possible Ranges of Gains
Constraints
Performance Equations
• So, we can formulate the solution as:

Minimize Minimize
Kp Ki Kd
Objective Function
Now We Solve it
PSO Example
 
PID Tuning Example (Solved)

1
• Using PID Controller for the given plant T.F.: 𝑇𝐹𝑝𝑙𝑎𝑛𝑡 =
𝑠 + 10
• T.F. of PID Controller is:
𝐾𝑖
𝑇𝐹𝑃𝐼𝐷 = 𝐾𝑝 + 𝐾𝑑 𝑠 +
𝑠
• Thus, the closed-loop T.F. is:

𝑘𝑑 𝑠 2 + 𝐾𝑝 𝑠 + 𝐾𝑖
𝐶𝐿𝑇𝐹 =
𝐾𝑑 + 1 𝑠 2 + 𝐾𝑝 + 10 𝑠 + 𝐾𝑖
• Therefore, the characteristic equation is:

𝐾𝑝 + 10 𝐾𝑖
𝑐ℎ. 𝑐𝑠 𝑒𝑞𝑛 = 𝑠 2 + 𝑠+ 𝑠 2 + 2𝜉ω𝑛 𝑠 + ω𝑛 2
𝐾𝑑 + 1 𝐾𝑑 + 1
PSO Example
 

• Using PSO to solve the tuning problem, with the following parameters:
• Swarm Size = 10 particles
w1 = 0.2
• Inertia weight = 0.2
• Personal best acceleration = 0.25 c1= 0.25
• Global best acceleration = 0.6 c2= 0.6
• Initial Velocities = 0 (for all particles)
• Use the objective function as:
Minimize the Overshoot percentage
• Number of Iterations = 100 Iterations
PSO Example
 

• Creating the initial swarm (Randomly Generated):
• Then Calculate the Fitness of each particle: Take Care:
The values below are not correct
Particle 01 78 8.3 0.12 53.264 %
Particle 02 23 5.2 0.05 34.126 %

The Best Particle
Particle 03 92 1.7 0.26 78.328 %
Particle 04 31 8.3 0.12 42.872 %

8 7.5 0.13
Particle 05 56 7.4 0.27 32.127 % This is the least value in this swarm,
so the particle is the closest to the
Particle 06 49 2.9 0.33 48.625 % food.
(Global Best)
Particle 07 29 9.3 0.16 27.851 %
Particle 08 41 6.3 0.48 32.251 % While, For each particle, this value
represent its current Personal best.
Particle 09 98 6.9 0.37 82.563 %
Particle 10 8 7.5 0.13 12.658 %
PSO Example
 

• Next, we move on to updating the velocities of each particle:
Use the rule:
Particle 01 𝑑
78 8.3 0.12 𝑉𝑖+1 = 𝑤1 ∗ 𝑉𝑖 + 𝑐1 𝑟1 ∗ 𝑃𝑏𝑒𝑠𝑡,𝑖 − 𝑋𝑖𝑑 + 𝑐2 𝑟2 ∗ 𝐺𝑏𝑒𝑠𝑡,𝑖
𝑑
− 𝑋𝑖𝑑
𝑤1 = 0.2, 𝑉𝑖 = 0
𝑐1 = 0.25, 𝑟1 = 0.25
𝑐2 = 0.6, 𝑟2 = 0.3
P_best 78 8.3 0.12

The personal best is the particle itself since
G_best 8 7.5 0.13 this is the first iteration.
(𝟎. 𝟐) 𝟎 + (𝟎. 𝟐𝟓) 𝟎. 𝟐𝟓 𝟕𝟖 − 𝟕𝟖 The only dominant factor will be the Global best,
+ (𝟎. 𝟔)(𝟎. 𝟑)(𝟖 − 𝟕𝟖) and the particles will start moving in its
direction.
Velocity @(i+1) -12.6 -0.144 0.0018
PSO Example
 

• Next, we move on to updating the positions of each particle:
Use the rule:
Particle 01 78 8.3 0.12 𝑑
𝑋𝑖+1 − 𝑋𝑖𝑑
𝑉𝑖+1 =
∆𝑇
Velocity @(i+1) -12.6 -0.144 0.0018 Given that ∆𝑇 is equal to 1 iteration (∆𝑇 = 1)
Thus,
𝑑
𝑋𝑖+1 = 𝑋𝑖𝑑 + 𝑉𝑖+1
1 Update the value of each element in the

𝑋𝑖+1 = 𝟕𝟖 + (−𝟏𝟐. 𝟔)
particle based upon initial position and
updated velocity.
And REPEAT for the rest of

65.4 8.156 0.1218
the particles in the swarm
The updated position of particle (01)
PSO Example


• Thus, the new swarm particles becomes:
• Then Calculate the Fitness of each particle: Take Care:
The values below are not correct
Particle 01 65.4 8.156 0.122 10.658 %
Particle 02 20.6 8.58 0.148 18.561 % The Global best

0.142 Particle becomes
Particle 03 16.4 8.22 17.532 %
Particle 04 23.7 7.59 0.135 25.631 % 65.4 8.156 0.1218
Particle 05 35.3 6.23 0.241 24.721 %

This is the least value in ALL swarms,
Particle 06 38.9 5.23 0.216 12.154 % so it becomes the global best
particle.
Particle 07 28.6 6.27 0.183 34.265 %
The Personal best is the best value
Particle 08 43.1 4.25 0.278 41.023 % for each particle in its history of
Particle 09 29.6 3.89 0.264 27.895 % motion.
Particle 10 8 7.5 0.13 12.658 % Then we repeat…

Next time ☺
Next time
• In the next Lecture we will cover:
– Ant Colony Optimization (ACO) Technique
• In the next tutorial / Lab we will cover:

– Particle Swarm Optimization (PSO) Examples
See you Next time … ☺

Engineering Optimization: Lecture

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Engineering Optimization: Lecture

Uploaded by

Copyright:

Available Formats

Ain Shams University (ASU)

Dr. Eng. Omar M. Shehata

Optimization Problem Formulation

Permutation Binary Arithmetic

Population based Optimization

Searching for Food

PSO Stopping Criteria

• All particles converge to • Each set of particles • The allowed number

Permutation Binary Arithmetic

Research Problem So we calculate the fitness of

PSO Operations: (1/2)

PSO Operations: (2/2)

PID Tuning Example

Control loop (PID) System response

PID Tuning Example

• So, we can formulate the solution as:

PID Tuning Example (Solved)

• Thus, the closed-loop T.F. is:

• Therefore, the characteristic equation is:

PID Tuning Example (Solved)

• Number of Iterations = 100 Iterations

PID Tuning Example (Solved)

Particle 02 23 5.2 0.05 34.126 %

Particle 04 31 8.3 0.12 42.872 %

PID Tuning Example (Solved)

P_best 78 8.3 0.12

Velocity @(i+1) -12.6 -0.144 0.0018

PID Tuning Example (Solved)

1 Update the value of each element in the

And REPEAT for the rest of

PID Tuning Example (Solved)

Particle 02 20.6 8.58 0.148 18.561 % The Global best

Particle 04 23.7 7.59 0.135 25.631 % 65.4 8.156 0.1218

Particle 05 35.3 6.23 0.241 24.721 %

Particle 10 8 7.5 0.13 12.658 % Then we repeat…

• In the next tutorial / Lab we will cover:

You might also like