Nghiên-cứu-khoa-học final-đã chuyển đổi PDF

ĐẠI HỌC THÁI NGUYÊN
TRƯỜNG ĐẠI HỌC KỸ THUẬT CÔNG NGHIỆP
BÁO CÁO TỔNG KẾT

ĐỀ TÀI KHOA HỌC VÀ CÔNG NGHỆ SINH VIÊN NĂM 2017
NGHIÊN CỨU CÀI ĐẶT THUẬT TOÁN TỰ HỌC ỨNG DỤNG

CHO ĐIỀU KHIỂN RÔ-BỐT
Mã số: SV 2017 - 04
Chủ nhiệm đề tài: Nguyễn Văn Hậu
Thái Nguyên- 2018

THAI NGUYEN UNIVERSITY OF TECHNOLOGY
FACULTY OF INTERNATIONAL TRAINING
FINAL REPORT
THEMES OF SCIENCE AND TECHNOLOGY STUDENT IN 2017
RESEARCH AND INSTALLATION OF LEARNING ALGORITHM

FOR ROBOT CONTROL
Code: 2017 - 04
Project Leader: Nguyen Van Hau
Thai Nguyen- 2018

ĐẠI HỌC THÁI NGUYÊN
TRƯỜNG ĐẠI HỌC KỸ THUẬT CÔNG NGHIỆP
NGHIÊN CỨU CÀI ĐẶT THUẬT TOÁN TỰ HỌC ỨNG DỤNG CHO ĐIỀU
KHIỂN RÔ-BỐT
Xác nhận của tổ chức chủ trì Giảng viên hướng dẫn
(chữ ký, họ tên)
Chủ nhiệm đề tài
(chữ ký , học tên)
Thái Nguyên- 2018

DANH SÁCH NHỮNG THÀNH VIÊN THAM GIA ĐỀ TÀI
Nội dung nghiên cứu cụ thể Ghi
TT Họ và tên Khoa chuyên môn,lớp
được giao chú
1 Nguyễn Văn Hậu Khoa Quốc tế, Chủ nhiệm đề tài:
Lớp K50APE
- Nghiên cứu tổng quan về trí
tuệ nhân tạo, rô-bốt tự học và
các ứng dụng tại Việt Nam và
trên thế giới
- Thiết kế, thử nghiệm sản phẩm
- Báo cáo đề tài
2 Trương Đức Tùng Khoa Quốc tế, Thành viên tham gia:
Lớp K50 APE - Nghiên cứu các thuật toán tự
học cho rô-bốt như thuật toán di
truyền (Genetic algorithm), thuật
toán bày chim (Particle swarm
optimization.
- Thiết kế sản phẩm
3 Quyền Văn Toán Khoa Quốc tế, Thành viên tham gia:
Lớp K50 APE - Thiết kế, chế tạo mô hình rô-bốt
tự học, mô hình điều khiển đơn
giản như điều khiển động cơ điện
1 chiều.
4 Lê Mỹ Anh Khoa Quốc Tế Thành viên tham gia:
Lớp K50 APE -Xây dựng thuật toán tự học và
ứng dụng thử nghiệm cho mô
hình rô-bốt.
- Thiết kế sản phẩm
- Kế toán ,báo cáo đề tài
4
LỜI CẢM ƠN
Lời đầu tiên, chúng tôi xin chân thành cảm ơn Trường Đại Học Kỹ Thuật Công Nghiệp
Thái Nguyên đã tạo điều kiện cho chúng tôi thực hiện đề tài nghiên cứu này.
Đề tài được hoàn thành dưới sự hướng dẫn của TS. Nguyễn Minh Ý. Chúng tôi xin gửi
tới Thầy lời biết ơn vô hạn về sự quan tâm giúp đỡ, tạo điều kiện tối đa để chúng tôi hoàn
thành dự án.
Thầy đã lên kế hoạch cho các cuộc họp hàng tuần để hỏi bất kỳ câu hỏi nào mà chúng tôi
phải đối mặt trong quá trình nghiên cứu. Thầy đã cung cấp cho chúng tôi các kiến thức cơ
bản về khoa học điện, công suất, phương pháp nghiên cứu và dạy chúng tôi cách sử dụng các
phần mềm như Mathlab, Arduino để mô phỏng những hoạt động của thuật toán liên quan đến
đề tài nghiên cứu. Và Thầy cũng đã giúp chúng tôi rất nhiều để hoàn thành các phần nghiên
cứu trong đề tài, hướng dẫn chúng tôi cách lập trình một số phần khó khăn.
Bên cạnh sự giúp đỡ của thầy hướng dẫn, chúng tôi xin cảm ơn các giáo viên khác đã hỗ
trợ chúng tôi hoàn thành dự án này. Cuối cùng nhưng không kém phần quan trọng, chúng tôi
muốn cảm ơn gia đình, bạn bè đã luôn ủng hộ chúng tôi về mọi mặt để hoàn thành đề tài
nghiên cứu một cách tốt nhất.
5
ACKNOWLEDGEMENTS
First of all, we would like to express our sincere thanks to Thai Nguyen University of
Technology for making this project possible.
This research was completed under the guidance of TS. Nguyen Minh Y. We would
like to extend our gratitude to him for his great interest in helping us complete this research
project.
He had planned for our meeting ones a week for asking any questions we have face in
the process of doing research. He has much knowledge about control algorithms, dynamic,
programing and power systems… He had provided us basic senses about electrical sciences,
control methods and taught us how to use the Mathlab software, Arduino to simulate things
related to our project. And also he helped me very much to accomplishing this thesis,
directed us how to write some difficult parts. Simply, I and my partner couldn’t looking
forward to a better and more enthusiasm supervisor .
Besides our advisor, we would like to thank other teachers for supporting us to finish
this project. Last but not least, we would like to thank our family, our friends for always
supporting us spiritually.
6
Contents
Tóm tắt ........................................................................................................................ 10
Abstracts...................................................................................................................... 11
PHẦN MỞ ĐẦU ......................................................................................................... 12
1. Tính Cấp Thiết Của Đề Tài .................................................................................. 12
2. Mục Tiêu Đề Tài .................................................................................................. 12
3. Tổng Quan Tài Liệu ............................................................................................. 13
3.1 Tình hình nghiên cứu ngoài nước .................................................................. 13
3.2 Tình hình nghiên cứu trong nước ................................................................... 14
4. Bố Cục Báo Cáo ................................................................................................... 14
CHAPTER 1 : THEORETICAL BASIS ................................................................. 15
1.1 Introduction ........................................................................................................ 15
1.2 Artificial intelligence techniques ........................................................................ 16
1.3 Several Learning Algorithms ............................................................................ 16
1.3.1 Genetic Algorithm (GA) ............................................................................. 16
1.3.2 Differential Evolution ................................................................................. 17
1.3.3 Ant Colony Optimization ............................................................................ 18
1.3.4 Tabu Search ................................................................................................. 18
CHAPTER 2: PSO ALGORITHM .......................................................................... 20
2.1 Introduction ........................................................................................................ 20
2.2 The algorithm ..................................................................................................... 21
2.3 Examples ............................................................................................................ 24
2.4 Advantages and Disadvantages. ......................................................................... 25
CHAPTER 3: SETTING PID COEFFICIENTS USING PSO ............................. 26
3.1 Introduction (PID) Controller ............................................................................. 26
3.2 Hardware Block Diagram ................................................................................... 32
3.3 Flowchart of PID value ...................................................................................... 36
3.4 Experiment.......................................................................................................... 37
CHAPTER 4 : ROBOT SETTING........................................................................... 40
7
4.1 Hardware components ........................................................................................ 40
4.1.1 Arduino Uno R3 .......................................................................................... 41
4.1.2 Servo Motors MG 996R .............................................................................. 42
4.1.3 LCD ............................................................................................................. 45
4.1.4 Encoder 400 Pulses MAX Mechanical Speed ............................................. 46
4.1.5 Basic Rubber Wheel .................................................................................... 48
4.1.6 Power Supply: Battery holder, 2 Ultrafire battery .................................... 49
4.2 How does the robot work ? ................................................................................. 50
4.2.1 Flowchart ..................................................................................................... 50
4.2.2 The working operation of crawling robot................................................... 51
4.3 Experiment And Result ...................................................................................... 52
4.3.1 Case 1 : Choose random angles ................................................................... 52
4.3.2 Case 2 : Using PSO algorithm ..................................................................... 52
CHAPTER 5 : CONCLUSION ................................................................................. 54
5.1 Orientation of the subject ................................................................................... 54
5.2 Conclusions ........................................................................................................ 54
5.3 Proposal for future work ..................................................................................... 55
REFERENCE ............................................................................................................. 56
APENDIX OF ROBOT SETTING ........................................................................... 58
APENDIX OF PSO-PID ............................................................................................ 61
8
LIST OF FIGURE
Figure 3. 1 A typical PID control structure .................................................................. 26
Figure 3. 2 Response of PV to step change of SP vs time, for three values of Kp ...... 27
Figure 3. 3 Response of PV to step change of SP vs time, for three values of Ki ....... 28
Figure 3. 4 Response of PV to step change of SP vs time, for three values of Kd ...... 29
Figure 3. 5 Effects of increasing a parameter independently ...................................... 30
Figure 3. 6 Block diagram of PID controller ............................................................... 31
Figure 3. 7 Calculation for sum of error ...................................................................... 31
Figure 3. 8 Block diagram of system .......................................................................... 32
Figure 3. 9 15V, 3 Amp Single Output Switching Power Supply ............................... 32
Figure 3. 10 Arduino MEGA 2560 .............................................................................. 33
Figure 3. 11 Technical specification of Arduino MEGA 2560 ................................... 33
Figure 3. 12 L298N Motor controller .......................................................................... 34
Figure 3. 13 DC Motor Gear Encoder 334 pluse ......................................................... 35
Figure 3. 14 Robot PSO-PID model .......................................................................... 35
Figure 3. 15 Flowchart of PSO-PID ............................................................................ 36
Figure 3. 16 Random values ......................................................................................... 37
Figure 3. 17 Experiment 5iters - 5members ................................................................ 37
Figure 3. 18 Experiment 10 iters – 10 members .......................................................... 38
Figure 3. 19 Experiment 25 iters – 40 members ......................................................... 38
Figure 3. 20 Results of PID experiment ...................................................................... 39
Figure 4. 1 Robot model .............................................................................................. 40

Figure 4. 2 Block Diagram of Robot ............................................................................ 40
Figure 4. 3 Arduino Uno R3......................................................................................... 41
Figure 4. 4 Technical specifications of Arduino Uno R3 ............................................ 41
Figure 4. 5 Servo Motors Mg 996r............................................................................... 42
Figure 4. 6 Servo motor interfacing with Arduino...................................................... 44
Figure 4. 7 LCD............................................................................................................ 46
Figure 4. 8 Encoder 400 Pulses MAX Mechanical Speed ........................................... 46
Figure 4. 9 Basic Rubber Wheel ................................................................................. 48
Figure 4. 10 Battery holder, 2 Ultrafire battery .......................................................... 49
Figure 4. 11 Flowchart of system 2 .............................................................................. 50
Figure 4. 12 The working operation of crawling robot ............................................... 51
Figure 4. 13 Experiment and result with random angles ............................................. 52
Figure 4. 14 Experiment and result using PSO algorithm .......................................... 52
9
Tóm tắt
Trong thời đại công nghiệp ngày nay, robot ngày càng được sử dụng phổ biến trong sản
xuất cũng như trong cuộc sống của con người. Các robot có vị trí quan trọng, rất khó để thay
thế, nó giúp mọi người làm việc trong điều kiện nguy hiểm, khó khăn. Ngoài ra, robot cũng
có thể được sử dụng trong lĩnh vực thăm dò không gian, quân sự, giải trí..v..v. Robot có thể
gánh vác trách nhiệm lớn hơn và chúng có thể được lập trình để tự hoạt động.
Trí thông minh nhân tạo là một lĩnh vực liên tục phát triển và cải thiện. Các nhà nghiên
cứu và nhà phát triển đang thiết kế các máy có khả năng cư xử, học hỏi và hành động giống
như con người. Thiết kế của các máy như vậy thường được nhắm mục tiêu để giải quyết các
vấn đề phức tạp một cách hiệu quả. Mục tiêu của nghiên cứu này là ứng dựng các thuật toán
tự học để thiết kế một rô bốt có thể học cách di chuyển tiến lên theo một đường thẳng liên
tục, không có bất kỳ hình thức hướng dẫn, giám sát hoặc kiến thức bên ngoài nào về hành vi
tự động của nó.
Thuật toán tự học , là một hình thức học tăng cường, được thực hiện trên hệ thống robot.
Trong giai đoạn học tập, các robot tương tác với môi trường của nó bằng cách ngẫu nhiên
bằng cách thử các chuyển động bằng cánh tay. Các cánh tay được kích hoạt bằng hai động cơ
servo. Đối với mỗi hành động được thực hiện, kết quả được tạo ra thông qua bộ mã hóa
quang học gia tăng. Mỗi kết quả là một dấu hiệu cho thấy giá trị đó hiệu quả đến mức nào.
Kết quả được tạo sau đó được sử dụng để tối ưu hóa chuỗi hành động tốt nhất thông qua
thuật toán PSO (Particle Swarm Optimization – Tối ưu hóa bày đàn ). Mục đích của thuật
toán là đạt được kết quả tối ưu bằng cách tối đa hóa các kết quả theo thời gian.
10
Abstracts
In today's Industrial Age, Robots are increasingly common use in the production as well
as in the lives of humans. The robots have an important position, which is difficult to
replace, it helps people to work in dangerous, hard conditions. In addition, the robot can also
be used in the field of space exploration, military, entertainment ect…They can shoulder
greater responsibilities and they can be programmed to manage themselves.
Artificial intelligence is an area which is continuously evolving and improving.

Researchers and developers are designing machines which are capable to behave, learn and
act like human beings. The design of such machines is often targeted to solve complex
problems in an efficient way. The aim of this thesis was to implement a crawling robot,
which is able to learn how to move forward in a continuous rectilinear path, without any
form of external guidance, supervision or prior knowledge of its dynamic behaviour.
The Learning algorithm, which is a form of reinforcement learning, was implemented on

the robotic system. During the learning phase, the robot interacts with its environment by
randomly trying possible arm actions. The arms are actuated using two servo motors. For
each performed action, a reward is generated via the incremental optical encoder. Each
reward is an indication of how good or how bad an action is. The generated rewards are then
used to optimise the best sequence of actions via the PSO-learning algorithm. The aim of the
algorithm is to achieve its optimal policy by maximising the rewards over time.
11
PHẦN MỞ ĐẦU
1. Tính Cấp Thiết Của Đề Tài
Cùng với sự phát triển của kinh tế - xã hội, nhân loại đang trải qua cuộc cách mạng
công nghiệp lần thứ 4: “Cách mạng 4.0” trong đó tập trung vào tự động hóa và trao đổi dữ
liệu trong công nghệ sản xuất. Nó bao gồm các hệ thống thực - ảo (Cyber-Physical Systems),
mạng lưới internet vận vật (Internet of Things – IoTs) và điện toán đám mây (Cloud
Computing). Trong cuộc cách mạng này, trí tuệ nhân tạo (Artificial Intelligence – AI) được
nhận định sẽ xuất hiện diện khác mọi lĩnh vực trong đời sống xã hội. Ví dụ như xe không
người lái đã được nghiên cứu và thử nghiệm trong nhiều lĩnh vực từ vận chuyển hang hóa,
khai thác mỏ đến chuyên chở người, v.v. Hay trong y học, với sự trợ giúp của trí tuệ nhân
tạo, bệnh nhân có thể dùng các ứng dụng trên điện thoại, chụp hình và điền vào các thông tin
gửi lên một hệ thống trí tuệ nhân tạo và gần như tức thì, kết quả chuẩn bệnh và cách điều trị
sẽ được trả về.
Trí tuệ nhân tạo đang dần đóng vai trò như một “trợ lý ảo” giúp con người chinh phục
những đỉnh cao tri thức mới, cải thiện năng suất lao động và đưa xã hội loài người phát triển
thêm một bước mới. Trong nhưng năm tới sẽ là kỉ nguyên của công nghệ robot, việc ứng
dụng trí tuệ nhân tạo vào việc điều khiển robot là một điều quan trọng và là xu thế tất yếu
của thế giới. Ngành công nghiệp robot và tự động hóa cùng với AI có thể thay thế lao động
trong tương lai không xa. Các nhà nghiên cứu cho rằng, có 50% khả năng các sản phẩm trí
tuệ nhân tạo sẽ tốt hơn con người trong tất cả các nhiệm vụ chỉ trong 45 năm nữa.
Với những luận giải như trên, chúng tôi đề xuất đề tài “Nghiên cứu cài đặt thuật toán tự
học ứng dụng cho điều khiển Rô-bốt”.
2. Mục Tiêu Đề Tài
- Nghiên cứu các thuật toán về trí tuệ nhân tạo và rô-bốt tự học như GA, PSO, Q-
learning, v.v.
- Nghiên cứu chế tạo thành công sản phẩm thực là mô hình rô-bốt tự học và ứng dụng các
thuật toán tự học.
12
- Giúp sinh viên ứng dụng các kiến thức đã được học trong chương trình đào tạo vào
nghiên cứu khoa học và làm việc với các hệ thống thực; nâng cao kiến thức, kỹ năng nghề
nghiệp; khả năng sử dụng thành thạo các công cụ, linh kiện và phần mềm hỗ trợ chuyên
ngành kỹ thuật điện.
- Đây là một lĩnh vực chưa được học tập và ứng dụng nhiều trong chương trình đào tạo,
là cơ hội cho sinh viên tìm hiểu, tiếp xúc và nghiên cứu một lĩnh vực mới trong tương lai.
3. Tổng Quan Tài Liệu
3.1 Tình hình nghiên cứu ngoài nước
Trên thế giới, rất nhiều ngành khoa học phải giải quyết các bài toán tối ưu đặc biệt là
trong ngành kỹ thuật. Nghiên cứu giải bài toán tối ưu là một vấn đề không đơn giản vì chưa
chắc có lời giải thỏa mãn đồng thời các mục tiêu đặt ra mà không vi phạm các ràng buộc nào
đó. Đã có nhiều phương pháp giải bài toán tối ưu được đề xuất, việc nghiên cứu phát triển
các phương pháp này và ứng dụng chúng để giải quyết các bài toán thực tế là một vấn đề
đang được quan tâm.
Trong thời gian gần đây, các kỹ thuật trí tuệ nhân tạo đã được thực hiện để khắc phục các
nhược điểm nêu trên của các kỹ thuật tối ưu hóa thông thường. Các kỹ thuật AI khác nhau
được xem xét để giải quyết vấn đề tối ưu hóa trong các hệ thống là: Genetic Algorithm
(GA), Swarm Optimization, Colony Optimization, Tabu Search, Simulated Annealing, etc,
vv… Kết hợp các kỹ thuật này, các kỹ thuật lai kết hợp các giá trị nhân tạo với các kỹ thuật
thông minh khác nhau. Những cải tiến trong các kỹ thuật trí tuệ nhân tạo này cũng đã được
đề xuất trong những năm qua.
Thuật toán tối ưu Particle Swarm Optimization (PSO) được R.C.Eberhat và J.Kennedy
đề nghị năm 1995. Từ lúc ra đời đến nay PSO đã được nhiều nhà khoa học tham gia nghiên
cứu, cải tiến và ứng dụng nó để giải nhiều bài toán lý thuyết và thực tế. Đề tài “Nghiên cứu
cài đặt thuật toán tự học ứng dụng cho điều khiển Rô-bốt” nhằm tìm hiểu khả năng ứng
dụng của thuật toán như PSO, GA trong việc giải quyết các bài toán tối ưu đa mục tiêu trong
thực tế.
13
3.2 Tình hình nghiên cứu trong nước
Không khó để nhận thấy sự quan tâm rất lớn của giới công nghê ̣ Việt Nam dành cho xu
hướng công nghệ tân tiến này.
Điề u đó thể hiện rất rõ qua kết quả tìm kiếm không ngừng tăng lên trên website Google
với những từ khóa công nghê ̣ 4.0 liên quan đế n “trí tuê ̣ nhân ta ̣o” (Artificial Intelligence),
“học máy” (Machine Learning), “dữ liệu lớn” (Big data) hay “va ̣n vâ ̣t kế t nố i” (Internet of
Things). Cụ thể, cụm từ "bắt đầu học machine learning như thế nào" trả về hàng trăm ngàn
kết quả, nhiều trong số đó là từ những trang web, blog đầy tâm huyết của các bạn trẻ Việt.
Nhiều người có chuyên môn tốt còn viết bài hướng dẫn rất hay trên các diễn đàn tiếng Anh,
được cộng đồng quốc tế nhiệt liệt tán dương.
Sự nắm bắt công nghệ mới nhanh nhạy của trí tuệ Việt Nam còn được quan sát thấy ở khả
năng không chỉ sử dụng thành thục những thiết bị công nghệ thông minh, tự cài đặt phần
mềm, mà còn năng động tham gia các cô ̣ng đồ ng trao đổi học thuật, cập nhật thông tin
nhanh chóng, thậm chí có thể tự sáng ta ̣o các thiế t bi ̣ như cánh tay robot thông minh điề u
khiể n bằ ng sóng não của các em học sinh , sinh viên.
4. Bố Cục Báo Cáo
Chương 1: Cơ sở lý thuyết , các thuật toán tự học thường được ứng dụng trong điều khiển
robot.
Chương 2: Tìm hiểu chuyên sâu về thuật toán PSO.
Chương 3: Tìm hiểu về cài đặt PID cho điều khiển robot.
Chương 4: Cài đặt thuật toán PSO cho điều khiển robot và xây dựng thiết kế robot ứng dựng
thuật toán PSO học bò một cách tự động
Chương 5: Kết luận
14
CHAPTER 1 : THEORETICAL BASIS
1.1 Introduction
A learning algorithm is a mathematical framework or procedure that calculates the best

output given a particular set of data. It does this by updating the calculation based on the
difference between the actual and desired output. These algorithms are typically concerned
with representation and generalization of the input data.
The fundamental task in robotics is to create an intelligent machine which achieves a

desired task through interaction with the world. The robot’s behaviour is reasoning,
acting and reacting based on knowledge and sensed information. Flexible behaviour
requires the ability to learn—to acquire knowledge, or improve skills, based on experience,
observation, or instruction.
Learning and robotics is an attractive combination. Our own intelligent behaviour
includes the ability to change our behaviour based on various forms of feedback. In
robotics we wish to emulate this adaptability in the hope of being able to achieve a
broader range of tasks in a more robust way. The idea has inspired various learning
systems for robots. However, implementing these systems has shown the difficulty of
applying current learning algorithms to robotics problems. To be practical, learning
algorithms need to become more robust to the effects of real sensors, real actuators, and
real time. The purpose of our research is to work towards a reinforcement learning
approach that is appropriate for robotics.
In this project, we will be applying the Learning algorithm to the Crawler in order for it
to learn how to move forward. The robot will pull itself forward using a two degree of
freedom arm by stepping through a state map as seen above.
The state map represents the various positions of the front and rear arm. As the robot
transitions to the right on the state map, the front arm rotates up. As it transitions down on
the state map, the rear arm rotates down. The robot will use the PSO Algorithm to learn what
is most optimum sequence of movements (or state transitions) in order to move forward.
15
These state transitions are rewarded, and are stored in the robot's representation of its own
reward map.
The wheels of the robot are not motorized. The right wheel simply has an encoder
attached to it in order for the robot to know how much it has moved forward. The algorithm
uses the input of the encoder in order to solve for the "quality" of the state transition. This
"quality" can either be positive or negative depending on the direction that the bot moved.
The algorithm also takes into account several other variables such as the previous quality of
the same transition, the level of random exploration of the state map, and more. The details
of the algorithm will be discussed later.
1.2 Artificial intelligence techniques
Artificial intelligence techniques try to simulate human behavior. They present

a better, faster and accurate solution to an optimization problem than the existing
conventional techniques. Artificial intelligence techniq ues generally make use of multiple
solutions to obtain an optimal solution. Some of the peculiar properties of these techniques
are:
- These methods remember past findings.
- They learn and adapt their performance.
- These methods can plan their path forward.
- These methods act intelligently according to human or social intelligence.
This section presents an overview of AI techniques for reactive power optimization
1.3 Several Learning Algorithms
1.3.1 Genetic Algorithm (GA)
GA uses the method of natural selection and genetics for obtaining an

optimized solution. David Goldberg (1989) has worked extensively on genetic
algorithms. The genetic algorithm search method is based on Darwin’s theory of
evolution. Genetic algorithm is a population based search method. In genetic algorithm
multiple solution are generated called as chromosomes. The chromosomes constitute
of random values of different control variables whose values are to be optimized. The
16
variation in these chromosomes values causes its movement which is utilized in the search
process.
The population of chromosomes is evaluated for the objective function. The
chromosomes having better values of objective function are chosen and retained through
selection process for next iteration. The chromosomes for next iteration are generated
through crossover and mutation of chromosomes of previous iteration. The chromosomes of
previous iteration are called as parents and the chromosomes generated for next iteration are
called as child. In the crossover process, a part of one of the parent chromosome is mixed
with another part of a different parent chromosome to generate a child chromosome.
In mutation process, some values of parent chromosome are altered to generate a
child chromosome. The new pool of child chromosomes is then evaluated for
objective function. These child chromosomes then act as parent chromosomes and the
process continues till the stopping criterion is met. The stopping criterion may be based
on number of iterations or convergence of control variable values stored in
chromosomes. GA is a rugged optimization technique with simple methodology. The
demerit of GA is it getting trapped in local minima.
1.3.2 Differential Evolution
Differential Evolution was introduced by Storn and Price in 1995. DE is similar to GA

except that it can efficiently handle the floating point and continuous values. DE generates a
random initial population, which then changes and improves through mutation, crossover
and selection processes applied through iterations. The main differences between the genetic
algorithm and DE algorithm are the selection process and the mutation scheme that makes
DE self adaptive . In the process of mutation, the existing solution is changed through a
scaled mutation factor. The crossover either merges the factors of two parents or a parent and
a random solution.
The selection process, selects the parent or child solution through tournament selection
process or any other similar process. The stopping criteria applied in DE is similar to that of
17
GA, it may be a certain number of iterations, or convergence of solutions or if the best
solution does not change for a specific number of iterations.
1.3.3 Ant Colony Optimization
Ant Colony Optimization (ACO) was introduced in 1992 by Dorigo. ACO is

based on foraging techniques of real ant colonies. ACO is a population based search, where
the ants search for food i.e., optimized solution. Artificial ants follow a construction
procedure for stochastic solution. A solution is built by adding iteratively the components
of a solution to incomplete solutions considering (i) any heuristic information about the
problem being solved, and (ii) pheromone trails. P heromone trails are present on paths
visited by ants. These trails vanish with time to facilitate search in new unexplored areas.
Ants do not move arbitrarily on construction graph, but rather follow a construction policy
which is a function of the problem constraints.
In general, ants try to build feasible solutions, but, if necessary, they can generate
infeasible solutions . In ACO, the colony of ants moves through adjacent solutions of the
optimization problem by construction of paths. The movement of ants also builds solution
of the optimization problem. Daemon actions are performed by the ants to execute
centralized actions which cannot be performed by a single ant. Poor computational
performance of ACO forms the main drawback of this technique .
1.3.4 Tabu Search
Tabu Search (TS) is a kind of neighborhood search. It has been mainly propagated
by Fred Glover. TS starts with a random solution and then neighborhood of current solution
is searched. The change in solution value from one iteration to other is called as a move. As
the current solution moves, in each iteration, the neighborhood also changes. The possible
neighborhood search is restricted by the number of solutions and the constraints of the
optimization problem. The restrictions on the movement of a solution are also placed due to
the ―Tabu List‖. Tabu list contains a set of moves which are prohibited. The list generally is
used to prevent cycling and it removes the chances of searching a solution which has been
previously visited. The size of tabu list plays an important role in optimization of problem.
18
The tabu list keeps changing as the search process progresses. The older entries in the tabu
list are eliminated with newer entries. An aspiration criterion may also be introduced to
override the tabu list, which helps achieve better results. The search process can be stopped
after some predefined number of iterations or if the optimum result does not change for a
certain number of iterations. TS on its own in not efficient to solve power optimization
problem, it should be combined with some other search process to enhance the capability of
the search process.
Abido has successfully implemented Tabu Search to solve the reactive power flow
control problem.
19
CHAPTER 2: PSO ALGORITHM
2.1 Introduction
Particle swarm optimization (PSO) is a population based stochastic optimization

technique developed by Dr. Eberhart and Dr. Kennedy in 1995, inspired by social behavior
of bird flocking or fish schooling.
PSO shares many similarities with evolutionary computation techniques such as

Genetic Algorithms (GA). The system is initialized with a population of random solutions
and searches for optima by updating generations. However, unlike GA, PSO has no
evolution operators such as crossover and mutation. In PSO, the potential solutions, called
particles, fly through the problem space by following the current optimum particles. The
detailed information will be given in following sections.
Compared to GA, the advantages of PSO are that PSO is easy to implement and there
are few parameters to adjust. PSO has been successfully applied in many areas: function
optimization, artificial neural network training, fuzzy system control, and other areas where
GA can be applied. In computational science, particle swarm optimization (PSO) is a
computational method that optimizes a problem by iteratively trying to improve a candidate
solution with regard to a given measure of quality.
Particle Swarm Optimization might sound complicated, but it's really a very simple
algorithm. Over a number of iterations, a group of variables have their values adjusted closer
to the member whose value is closest to the target at any given moment. Imagine a flock of
birds circling over an area where they can smell a hidden source of food. The one who is
closest to the food chirps the loudest and the other birds swing around in his direction. If any
of the other circling birds comes closer to the target than the first, it chirps louder and the
others veer over toward him. This tightening pattern continues until one of the birds happens
upon the food. It's an algorithm that's simple and easy to implement.
The algorithm keeps track of three global variables:

➢ Target value or condition
20
➢ Global best (gBest) value indicating which particle's data is currently closest to the Targe.
➢ Stopping value indicating when the algorithm should stop if the Target isn't found
Each particle consists of:
➢ Data representing a possible solution
➢ A Velocity value indicating how much the Data can be changed
➢ A personal best (pBest) value indicating the closest the particle's Data has ever come to
the Target.
2.2 The algorithm
As stated before, PSO simulates the behaviors of bird flocking. Suppose the following
scenario: a group of birds are randomly searching food in an area. There is only one piece of
food in the area being searched. All the birds do not know where the food is. But they know
how far the food is in each iteration. So what's the best strategy to find the food? The
effective one is to follow the bird which is nearest to the food.
PSO learned from the scenario and used it to solve the optimization problems. In PSO,
each single solution is a "bird" in the search space. We call it "particle". All of particles have
fitness values which are evaluated by the fitness function to be optimized, and have
velocities which direct the flying of the particles. The particles fly through the problem space
by following the current optimum particles.
PSO is initialized with a group of random particles (solutions) and then searches for
optima by updating generations. In every iteration, each particle is updated by following two
"best" values. The first one is the best solution (fitness) it has achieved so far. (The fitness
value is also stored.) This value is called pbest. Another "best" value that is tracked by the
particle swarm optimizer is the best value, obtained so far by any particle in the population.
This best value is a global best and called gbest. When a particle takes part of the population
as its topological neighbors, the best value is a local best and is called lbest.
After finding the two best values, the particle updates its velocity and positions with
following equation (a) and (b)
21
• Algorithm
[x*] = PSO()
P = Particle_Initialization(); For
i=1 to it_max
For each particle p in P do fp = f(p);
If fp is better than f(pBest)
pBest = p;
end
end
gBest = best p in P;
For each particle p in P do
v = v + c1*rand*(pBest – p) + c2*rand*(gBest – p);
p = p + v;
end
end
where
p: particle‟s position
v: path direction
c1: weight of local information
c2: weight of global information
pBest: best position of the particle
gBest: best position of the swarm
rand: random variable
22
1. Create a „population‟ of agents (particles) uniformly distributed over X.
2. Evaluate each particle‟s position according to the objective function.
3. If a particle‟s current position is better than its previous best position, update it.
4. Determine the best particle (according to the particle‟s previous best positions).
5. Update particles velocities:
vi (t +1) = vi (t) + c1r1[xi (t) − xi (t)] + c2 r2 [g(t) − xi (t)]
where:
✓ i is the partical index
✓  is the inertial coefficient 0 ≤ c1, c2 ≤ 2
✓ r1, r2 are random values ( 0 ≤ r1, r2 ≤ 1)regenerated every velocity update
✓ vi (t) is the particle‟s velocity at time t
✓ xi (t) is the particle‟s position at time t

✓ xi (t) is the particle‟s individual best solution as of time t
✓ g(t) is the swarm‟s best solution as of time t
✓  Inertia coefficient usually between 0.8 and 1.2
✓ Cognitive coefficient c1
✓ Social coefficient c2
• Update position:
xi (t +1) = xi (t) + vi (t +1)
Such a formulation was intended to impose some restriction on velocity vectors and
thus to prevent divergence. Eberhart and Shi compared the use of inertia weights and
constriction factors in particle swarm optimization and concluded that the two approaches
are equivalent and could be interchangeably used by proper parameter setting. They also
indicated that the use of constriction factor does not eliminate the need for Vmax parameter
23
unlike what might be assumed at the first glance. Though the two approaches are shown to
be equivalent, they both survived and have been continually used by researcher.
2.3 Examples
24
Figure 2.1 Example of Swarm in PSO Algorithm
2.4 Advantages and Disadvantages.
• Advantages
➢ Insensitive to scaling of design variables
➢ Simple implementation
➢ Easily parallelized for concurrent processing
➢ Derivative free
➢ Very few algorithm parameters
➢ Very efficient global search algorithm
• Disadvantages
➢ Tendency to a fast and premature convergence in mid optimum points
➢ Slow convergence in refined search stage (weak local search ability)
25
CHAPTER 3: SETTING PID COEFFICIENTS USING PSO
3.1 Introduction (PID) Controller
A PID Controller is a common technique used to control a wide variety of machines

including vehicles, robots and even rockets. The complete mathematical description of a PID
Controller is fairly complex but a much simpler understanding is really all that is needed to
use a PID effectively.
A typical structure of a PID system is shown, where it can be seen that in a PID
controller, the error signal e(t) is used to generate the proportional, integral, and derivative
actions, with the resulting signals weighted and summed to form the control signal u(t)
applied to the plant model. A mathematical description of the PID controller is:
Figure 3. 1 A typical PID control structure

t
de(t )
u (t ) = MV (t ) = K p e(t ) + K  e( )d + K d
0
dt
✓ Kp is the proportional gain, a tuning parameter

✓ Ki is the integral gain, a tuning parameter
✓ Kd is the derivative gain, a tuning parameter,
✓ e(t) = SP- PV(t) is the error (SP is the setpoint, and PV(t) is the process
variable),
✓ t is the time or instantaneous time (the present),
✓ ґ is the variable of integration (takes on values from time 0 to the present t).
26
• Proportional term
The proportional term produces an output value that is proportional to the current error
value. The proportional response can be adjusted by multiplying the error by a constant Kp,
called the proportional gain constant.
The proportional term is given by:
Pout = K pe(t)
A high proportional gain results in a large change in the output for a given change in
the error. If the proportional gain is too high, the system can become unstable (see the
section on loop tuning). In contrast, a small gain results in a small output response to a
large input error, and a less responsive or less sensitive controller. If the proportional gain
is too low, the control action may be too small when responding to system disturbances.
Tuning theory and industrial practice indicate that the proportional term should contribute
the bulk of the output change.
Figure 3. 2 Response of PV to step change of SP vs time, for three values of Kp
27
• Integral term
The contribution from the integral term is proportional to both the magnitude of the
error and the duration of the error. The integral in a PID controller is the sum of the
instantaneous error over time and gives the accumulated offset that should have been
corrected previously. The accumulated error is then multiplied by the integral gain (Ki) and
added to the controller output.
t
The integral term is given by: I = K i  e( )d
0
Figure 3. 3 Response of PV to step change of SP vs time, for three values of Ki

The integral term accelerates the movement of the process towards set point and
eliminates the residual steady-state error that occurs with a pure proportional controller.
However, since the integral term responds to accumulated errors from the past, it can cause
the present value to overshoot the set point value (see the section on loop tuning).
28
• Derivative term
The derivative of the process error is calculated by determining the slope of the error
over time and multiplying this rate of change by the derivative gain Kd. The magnitude of
the contribution of the derivative term to the overall control action is termed the derivative
gain, Kd.
The derivative term is give by:
de(t )
Dout = K d
dt
Figure 3. 4 Response of PV to step change of SP vs time, for three values of Kd
Derivative action predicts system behavior and thus improves settling time and stability of
the system. An ideal derivative is not causal, so that implementations of PID controllers
include an additional low-pass filtering for the derivative term to limit the high-frequency
gain and noise. Derivative action is seldom used in practice though – by one estimate in
only 25% of deployed controllers–because of its variable impact on system stability in real-
world applications.
29
• Manual turning
If the system must remain online, one tuning method is to first set Ki and Kd values
to zero. Increase the Kp until the output of the loop oscillates, then the Kp should be set to
approximately half of that value for a "quarter amplitude decay" type response. Then
increase Ki until any offset is corrected in sufficient time for the process. However, too
much Ki will cause instability. Finally, increase Kd, if required, until the loop is acceptably
quick to reach its reference after a load disturbance. However, too much Kd will cause
excessive response and overshoot. A fast PID loop tuning usually overshoots slightly to
reach the set point more quickly; however, some systems cannot accept overshoot, in which
case an overdamped closed-loop system is required, which will require a Kp setting
significantly less than half that of the Kp setting that was causing oscillation.
Figure 3. 5 Effects of increasing a parameter independently
• Error of PID
A PID controller continuously calculates an error value e(t) as the difference between a
desired setpoint (SP) and a measured process variable (PV) and applies a correction based on
proportional, integral, and derivative terms (denoted P, I, and D respectively), hence the
name.
30
Figure 3. 6 Block diagram of PID controller
Integral calculation based on sum of error e(t ) = SP − PV . Integration of error means that we continually
sum controller error up to the present time.
In our code, we declare e(t)=RMSE
RMSE = integral of absolute( error)
Figure 3. 7 Calculation for sum of error
➢ The smaller the RMSE, the faster we can reach the desired value.
31
3.2 Hardware Block Diagram
PSO
PID
Power Motor DC
Controller
Supply Driver Motor
Figure 3. 8 Block diagram of system
• Block diagram Description

− 15V, 3 Amp Single Output Switching Power Supply
• Specifications:
✓ Case Size: 111 x 78 x 36mm

✓ Input Frequency: 50-60Hz
✓ 15V Power Supply
✓ 3A Single Output
✓ AC: 110/220V
✓ Power 45W
✓ Protection: Overload |
Overpressure Short circuit.
✓ Cooling: air convection
✓ Safety Compliance: CCC / FCC /
CE
✓ Working temperature: 0 ~ 40 ° C
✓ Storage temperature: -20 ~ 60 ° C
✓ Environmental humidity: 0 ~ 9
Figure 3. 9 15V, 3 Amp Single

Output Switching Power Supply
32
− Controller (Arduino MEGA 2560)
Figure 3. 10 Arduino MEGA 2560
• Technical specifications
Microcontroller ATmega 2560

Operating Voltage 5V
Input Voltage 7-12V
(recommended)
Input Voltage (limit) 6-20V
Digital I/O pins 54(of which 15 provide PWM output)
Analog Input Pins 16
DC Current per I/O Pin 20mA
DC Current for 3,3V Pin 50mA
Flash Memory 256kB of which 8kB used by
bootloader
SRAM 8kB
EEPROM 4kB
Clock Speed 16MHz
LED_BUILTIN 13
Length 101,52mm
Width 53.3mm
Weight 37g
Figure 3. 11 Technical specification of Arduino MEGA 2560

33
− Motor Driver (L298N)
L298N Motor Driver IC is used in this project to drive the motors of the
robot. It receives signals from Arduino.
• Specifications:
✓ Module name : dual H bridge motor
drive module
✓ Work mode : H bridge drive (double
road)
✓ Main control chip : L298N
✓ Packaging : electrostatic bag
✓ Logical voltage : 5V
✓ Drive voltage : 5v-35v
✓ Logical current : 0mA-36mA
✓ Driving current : 2A(MAX single
bridge)
✓ Maximum power : 25W
✓ Weight : 30 g
✓ Peripheral dimensions : 43*43*27mm
Figure 3. 12 L298N Motor controller
34
− 334 Pulse Encoder Motor + RP463 RPM Reducer 1: 9
Figure 3. 13 DC Motor Gear Encoder 334 pluse
Figure 3. 14 Robot PSO-PID model

35
3.3 Flowchart of PID value
Figure 3. 15 Flowchart of PSO-PID
36
3.4 Experiment
Random values kp = 0.7, ki = 0.1, RMSE= 8678
Figure 3. 16 Random values
• 5iters - 5members
ki_best = 0.010996, kp_best = 0.010658
Figure 3. 17 Experiment 5iters - 5members
37
• 10 iters – 10 members
ki_best = 0.016003, kp_best = 0.02637
Figure 3. 18 Experiment 10 iters – 10 members
• 25 iters – 40 members
ki_best = 0. 0203, kp_best = 0. 0385
Figure 3. 19 Experiment 25 iters – 40 members
38
3.5 Results
Kp Ki RMSE Time for trial
Random values 0.1 0.07 8678
10 members, 10 0.0106580 0.010996 5646 125

iters
40 members, 25 0.0385 0.0203 2634 5000
iters
Figure 3. 20 Results of PID experiment

➢ Classical PID controller will be better when PSO is applied
➢ The greater numbers of members and iterations, the greater result
✓ ki_best = 0.0203
✓ kp_best = 0.0385
✓ RMSE= 2634
39
CHAPTER 4 : ROBOT SETTING
4.1 Hardware components
Figure 4. 1 Robot model
• 1x Arduino Uno
• 2x Servo Motors
• 1x Encoder 400 Pulse
• 2x Wheels
• 2x 3.7V Battery
• 1x LCD
Figure 4. 2 Block Diagram of Robot
40
4.1.1 Arduino Uno R3
The UNO is the best board to get started

with electronics and coding. The UNO is
the most used and documented board of
the whole Arduino family.
Figure 4. 3 Arduino Uno R3
• Technical specifications
Microcontroller ATmega328P
Operating Voltage 5V
Input Voltage (recommended) 7-12V
Input Voltage (limit) 6-20V
Digital I/O Pins 14 (of which 6 provide PWM output)
PWM Digital I/O Pins 6
Analog Input Pins 6
DC Current per I/O Pin 20mA
DC Current for 3.3V Pin 50mA
Flash Memory 32 KB (ATmega328P) of which 0.5
KB used by bootloader
SRAM 2 KB (ATmega328P)
EEPROM 1 KB (ATmega328P)
Clock Speed 16 MHz
LED_BUILTIN 13
Length 68.6 mm
Width 53.4 mm
Weight 25 g
Figure 4. 4 Technical specifications of Arduino Uno R3
41
4.1.2 Servo Motors MG 996R
MG996 Servo Motors are the most commonly used type in robot designs or car
navigation. The MG996 servo motor has a strong tug, and the gears and gears are made
entirely of metal so they have high durability. The engine is built-in. The driver controls
the internal engine according to the pulse-rotation mechanism so it is very easy to use.
Figure 4. 5 Servo Motors Mg 996r
✓ Size: 40.7x19.7x42.9 (mm)

✓ Operating speed (4.8V no load): 0.17sec/60 degrees
✓ Operating speed (6.0V no load): 0.14sec/60 degrees
✓ Stall torque (4.8V): (9.4kg/cm)
✓ Stall torque (6.0V): (11kg/cm)
✓ Temperature range: -30 to +60?
✓ Dead band width: 5usec
✓ Operation Voltage: 4.8V-7.2V
✓ Item weight: 55g
42
Inside most Servo Motors you will find: A motor, gears, some type of limit stops that
will limit the movement of the shaft, a potentiometer of some kind for position feedback
and some integrated circuit to move the servo to a specific position.
A Standard Servo has around 180 degrees of motion. These can be modified to make
them rotate 360 degrees or you can buy them already made this way.
The modification involves opening the Servo case, removing the limiting device and
disconnect the potentiometer from the shaft. Once done the Servo will rotate in either
direction endlessly since it has no way of knowing it’s position anymore and there are no
limit switches to stop it.
Most Servo Motors have three wires:
Black: Ground
Red: Voltage
White or Yellow: Control Wire.
To move a Servo you send a pulse to the control wire. This process is referred to
Pulse Coded Modulation. A servo motor is a rotary actuator mostly coupled with a shaft
or arm. It has a position control associated with the servo mechanism. It uses a closed-
loop control system with error-sensing negative feedback to correct the performance of a
mechanism. A servo motor maintains accurate control of angular position and speed of
motion of the rotor. The Servos have integrated drive gears and circuit, which precisely
control the servo position.
The angular positions of a servo motor are controlled by the pulse width
modulation(PWM).
The input pulse train at the control signal will turn the rotor to the desired position.
The servo motor expects a control pulse at every 20 milliseconds (ms). The width of each
pulse directs the servo to turn the shaft to the corresponding angular positions. That is the
duration of the positive pulse in a 20ms total pulse width determines the servo shaft
43
position. For a standard servo, 1ms positive pulse maintains a 0° and a maximum of the
2ms positive pulse will have a position of 180°.
Thus the pulse width between 1ms and 2ms obtains a corresponding position
between 0° to 180° angles respectively.
4.1.2.1 Servo motor interfacing with Arduino
The servo motors are interfaced with the Arduino through a standard three-wire
connection.
Figure 4. 6 Servo motor interfacing with Arduino

• Power
The power wire mostly has a red colour, which connects to the 5V pin of the
Arduino.
The Servo motor requires a considerable amount of power, especially for high power
servos. So, for multiple servos or while using servos along with other pins, it is better to
power the servo motor separately with an external supply. Because the power at the
remaining pins would be interrupted during its operation.
For external powering, connect the ground of the arduino commonly with the -ve
terminal of the external power supply. And connect the supply terminal of the servo (+V)
to the +Ve terminal of the external supply.
44
• Ground
The ground wire typically has black or brown colour. It connects to the ground pin of
the Arduino.
• Control signal
The signal wire commonly has an orange colour. Yellow, white, blue colours are also
used for this connection. One of the Digital pins of Arduino can be used for the signal
connection. But, it is commonly used on PWM pins (3, 5, 6, 9, 10, or 11). The servo is
mostly connected to pin 9 on the Arduino board. Even the servo is not in use,
the analogWrite () (PWM) functionality on pins 9 and 10 disables by the accessing of the
library function (except the Arduino Mega). Thus the remaining PWM pins can use for
analogwrite () (PWM) by connecting servos to the pin 9 or 10.
• Serial monitor speed control of a sweeping servo
This Arduino sketch is to control the position and speed of a servo motor using
Arduino serial communication. Here the input decimal values of position and the speed of
the shaft movement can be given through the serial monitor. The values can be separated
by entering the position and the time delay by a comma.
Here we are controlling the speed of a servo motor moving in a sweeping movement.
The speed of sweeping movement can be controlled by varying the time delay in the “for
{}” loop. As it adds the time delay between each loop, that is in each increment (from 0°
to 180°) or decrements (from 180° to 0° ) in the value of an angle. Thus the servo shaft
will have a time delay in each angular position, causes the variation in speed of the shaft.
Here the speed of the servo can be adjusted by the serial input. The time delay is
entered as decimal values in milliseconds
4.1.3 LCD
45
Figure 4. 7 LCD
It has 16 pins and the first one from left to right is the Ground pin. The second pin is
the VCC which we connect the 5 volts pin on the Arduino Board. Next is the Vo pin on
which we can attach a potentiometer for controlling the contrast of the display.
Next, The RS pin or register select pin is used for selecting whether we will send
commands or data to the LCD. For example if the RS pin is set on low state or zero volts,
then we are sending commands to the LCD like: set the cursor to a specific location, clear
the display, turn off the display and so on. And when RS pin is set on High state or 5
volts we are sending data or characters to the LCD.
4.1.4 Encoder 400 Pulses MAX Mechanical Speed
Figure 4. 8 Encoder 400 Pulses MAX Mechanical Speed
46
• Features :
✓ 400 p/r (Single-phase 400 pulses /R,Two phase 4 frequency doubling to 1600
pulses)
✓ Power source: DC5-24V
✓ Shaft: 6*13mm/0.23*0.51"
✓ Size: 38*35.5mm/1.49*1.39"
✓ Output :AB 2phase output rectangular orthogonal pulse circuit, the output for
the NPN open collector output type
✓ Maximum mechanical speed: 5000 R / min
✓ Response frequency: 0-20KHz
✓ Cable length: 1.5 meter
✓ Notice:AB 2phase output must not be directly connected with VCC, otherwise,
will burn the output triode
✓ Because different batches, and may not have the terminal
4.1.4.1 How to run Rotary Encoder + Arduino

A rotary encoder is a device that you can rotate infinitely. Simple ones have no real
state like a pot does, so when you start up, you won’t be able to simply read from the
encoder where it is turned to. But because you can keep turning it it has no beginning,
middle or end anyways. However, if you keep track of that rotation in code, you can use
it as a knob input you can turn up or down as much as you would like.
Most simple encoders like this only make use of 4 pins, one of those is ground and
other Vcc. Those other two pins change state and are always either high or low, so they
can only have a total of 4 combinations. 00, 01, 10, and 11. This is known as 2 bit gray
code. So when you turn it, the arduino can say… Well you were at 01, and now you are at
00 so you move this way. Or you were at 01, but now you are at 10 so you must have
47
moved the other way. You can see that this encoder has 5 pins, the other 2 are just a
simple switch that is engaged when you press down. (see the second illustration on the
right)
It sounds super simple, and it is kind, but what we can do is every time a value
changes we can check what direction it moved. Then if we increment a value every time
it turned one way, and deincrement it when we move one step the other, we can keep
track of how much it has moved since we started. So if you want a knob that can turn up
to 11, this is your guy.
So, the really funky thing about a rotary encoder is for it to work, we need to know
every time those values change. This can be hard because if the arduino is in the middle
of doing something, like delay(1000) or what have you, we will miss the change. So we
need a way to say to the arduino “I don’t care what you are doing, or when you are doing
it, if you see any of these two pins change state, you drop everything and attend to them”.
To do this we need something called interrupts.
4.1.5 Basic Rubber Wheel

These are a pair of basic, 65mm wheels with black rubber tires. These wheels are the
same ones designed to fit onto DAGUs right angle gear motors. With these wheels you
will be able to get almost any beginners robot project moving.
These wheels are sold in packs of two.
Figure 4. 9 Basic Rubber Wheel
48
4.1.6 Power Supply: Battery holder, 2 Ultrafire battery
Figure 4. 10 Battery holder, 2 Ultrafire battery

The vehicle is powered by two (9 volts) batteries as a primary source of an electrical
energy for the motors and as a power supply for the Arduino
49
4.2 How does the robot work ?
4.2.1 Flowchart
Figure 4. 11 Flowchart of system 2
50
First we initialize the initial random angle values .Initialization of the PSO member
and iteration.
The algorithm then calculates the values and gives Delta distance. If Delta distance is
greater than P best value, then we assign that value to the best value including P best
value and P Best angel 1, P best angel 2.
If Delta distane is less than P best value , we will compare with G best value . Delta
distance is greater than G best value , We also assign G best value equal to Delta distance
and G best angle 1 = Position Angle 1 , G best angle 2 = Position Angle 2.
Then , We update all the angles (which we have generated from beginning ) by
PSO algorithsm with new iteration and find the best value after the last iteration .
4.2.2 The working operation of crawling robot
Figure 4. 12 The working operation of crawling robot
The robot has to move to the front, but the agent does not have any knowledge about the
environment previously. At each time step, the agent observes noisy controller-readings
of the joint angles, and outputs turning direction of the joint motors. The immediate
reward is defined as the distance of the body movement by the step. In order to move the
body to the front, the arm should be controlled as if it is crawling. Through trial and error,
the agent has to learn such a control policy that maximizes reward function. The
environment is considered to be a partially observable decision processes which has
continuous state space .
51
4.3 Experiment And Result
4.3.1 Case 1 : Choose random angles
Angle 1 (degree) Angle 2(degree) Distance

(Number of pulse )
50 70 0
100 80 23
120 90 86
150 100 124
Figure 4. 13 Experiment and result with random angles

In those case we choose randomly the angle for the servo,the distance is gotten
from each single step, and the result is not good, even the robot can not swam, so it’s
very hard to determine the optimal angles to get the maximum distance.
It can be found out the optimal angles, but obviously take a lot of time.
4.3.2 Case 2 : Using PSO algorithm
Members Iter Distance Angle 1 Angle 2 eoTime for

(Number of pulse ) (degree) (degree) trial
(second)
3 3 279 130 92 16
10 3 332 133 100 60
20 3 334 153 100 121
3 8 378 150 100 48
10 8 407 158 100 160
20 8 411 160 100 320
Figure 4. 14 Experiment and result using PSO algorithm
52
That is the result when we use the PSO algorithm, the distance is gotten from each single
step, the optimal angles 1 and 2 found out by the corresponding number of members and
iterations with the whole learning time at the last column. It’s easy for robot can go and
take very short time. Only by several iterations we can find out the optimal angles for
both servos.
The result depends on how many members and iterations, the more the members, the
more chances we can find out the best angles. But it means the system need more time.
53
CHAPTER 5 : CONCLUSION
5.1 Orientation of the subject
A crawling robot is used in engineering subjects. It is used to test many different
self-learning algorithms in artificial intelligence.
From self-learning how to crawl, robots can learn many deeper evolutionary steps
like learning to go, learning to run. From there we can apply more self-learning
algorithms to robot control such as Genetic algorithm, Particle Swarm Optimization, Ant
Colony Optimization, Tabu Search, Simulated.
5.2 Conclusions
This paper presents a novel design method for determining the PID controller
parameters using the PSO method. The proposed method integrates the PSO algorithm
with the new time-domain performance criterion into a PSO-PID controller.
From the results, the designed PID controller using PSO algorithm shows superior
performance over the traditional method of Ziegler-Nichols, in terms of the system
overshoot, settling time and rise time. However, the traditional method provides us with
the initial PID gain values for optimal tuning. Therefore the benefit of using a modern
artificial intelligence optimization approach is observed as a complement solution to
improve the performance of the PID controller design conventional method. Of course
there are many techniques can be used as the optimization tools and PSO is one of the
recent and efficient optimization tools.
The major drawback of PSO is that the movement of particles is based on
inertia weights and constants. The values of these constants have been widely studied
and may vary according to the problem being studied.
The swarm size in PSO remains constant. The swarm size is supposed to change
during the process of the search method. It also introduces neighborhood search and
eliminates the affect of best position particles as the particle itself moves to better
positions.
54
The aim of this thesis was to implement a crawling robot, which is able to learn how
to move backward or forward in a continuous rectilinear path, without any form of
external guidance, supervision or prior knowledge of its dynamic behaviour. The PSO-
learning algorithm, which is a form of reinforcement learning, was implemented on the
robotic system. During the learning phase, the robot interacts with its environment by
randomly trying possible arm actions.
The arms are actuated using two servo motors. For each performed action, a reward is
generated via the incremental optical encoder. Each reward is an indication of how good
or how bad an action is. The generated rewards are then used to optimize the best
sequence of actions via the PSO-learning algorithm. The aim of the algorithm is to
achieve its optimal policy by maximizing the rewards over time. Finally the optimal arm
angles and sequence are identified, thus enabling the required forward movements.
5.3 Proposal for future work
Machine learning offers to robotics a framework and set of tools for the design of
sophisticated and hard-to-engineer behaviors; conversely, the challenges of robotic
problems provide both inspiration, impact, and validation for developments in robot
learning. The relationship between disciplines has sufficient promise to be likened to that
between physics and mathematics.
In this project, we attempt to strengthen the links between the two research
communities by providing a survey of work in robot learning for learning control and
behavior generation in robots. We highlight both key challenges in robot learning as well
as notable successes. We discuss how contributions tamed the complexity of the domain
and study the role of algorithms, representations, and prior knowledge in achieving these
successes. As a result, a particular focus of our project lies on model learning for control
and robot reinforcement learning.
We hope how machine learning approaches may be profitably applied, and we note
throughout open questions and the tremendous potential for future research.
55
REFERENCE
[1] Yuhui Shi, Russell C. Eberhart, "Empirical Study of Particle Swarm
Optimization", Evolutionary Computation, CEC 99, Vol 3 , 6-9 July 1999
[2] Wen Zhang, Yutian Liu, "Reactive Power Optimization Based on PSO in a Prac
tical Power System", IEEE Power Engineering Society General Meeting, Vol.1, 6-
10 June 2004, pp:239 - 243
[3] www.researchgate.net/publication/221438658_The_Crawler_A_Class_Room_De
monstrator_for_Reinforcement_Learning?
[4] https://en.wikipedia.org/wiki/Particle_swarm_optimization
[5] Kennedy, J.; Eberhart, R. (1995). "Particle Swarm Optimization". Proceedings of
IEEE International Conference on Neural Networks. IV. pp. 1942–
1948. doi:10.1109/ICNN.1995.488968.
[6] Shi, Y.; Eberhart, R.C. (1998). "A modified particle swarm
optimizer". Proceedings of IEEE International Conference on Evolutionary
Computation. pp. 69–73.
[7] Kennedy, J. (1997). "The particle swarm: social adaptation of
knowledge". Proceedings of IEEE International Conference on Evolutionary
Computation. pp. 303–308.
[8] Poli, R. (2007). "An analysis of publications on particle swarm optimisation
applications" (PDF). Technical Report CSM-469. Department of Computer
Science, University of Essex, UK.
[9] Bonyadi, M. R.; Michalewicz, Z. (2017). "Particle swarm optimization for single
objective continuous space problems: a review". Evolutionary
Computation. 25 (1): 1–54. doi:10.1162/EVCO_r_0018
[10] https://en.wikipedia.org/wiki/Servomotor
[11] https://www.jameco.com/jameco/workshop/howitworks/how-servo-motors-
work.html
[12] Arduino Motor Shield: http://arduino.cc/en/Main/ArduinoMotorShieldR3
chipKIT
56
[13] Uno32DevelopmentBoard: http://www.digilentinc.com/Products/Detail.cfm
?NavPath=2,892,893&Prod=CHIPKIT-UNO32 /
[14] Nedjah, N., Mourelle, L.D.M.: Systems Engineering Using Particle Swarm
Optimisation. Nova Science Publishers (2007)
[15] Shi, Y., Eberhart, R.: A Modified Particle Swarm Optimiser. In: IEEE
International Conference on Evolutionary Computation, pp. 69–73. IEEE Press,
Piscataway (1998)
[16] del Valle, Y., Venayagamoorthy, G.K., Mohagheghi, S., Hernandez, J.,
Harley, R.G.: Particle Swarm Optimisation: Basic Concepts, Variants and
Applications in Power Systems. IEEE Transactions On Evolutionary Computation
12(2), 171–195 (2008)
[17] Kennedy, J., Eberhart, R.C.: Particle Swarm Optimisation. In: Proceedings
of IEEE International Conference on Neural Networks, Perth, Australia, pp. 1942–
1948 (1995)
[18] Huang, C.M., Huang, C.J., Wang, M.L.: A Particle Swarm Optimisation to
Identifying the ARMAX Model for Short term Load Forecasting. IEEE
Transactions on Power Systems 20, 1126–1133 (2005)
57
APENDIX OF ROBOT SETTING
#include <math.h>
#include <Servo.h>
#include <LiquidCrystal.h>
LiquidCrystal lcd(12, 11, 7, 6, 5, 4);
#define PinServo_1 9
#define PinServo_2 10
Servo Servo_1;
Servo Servo_2;
int PulseNumber;
const int Chanel_A = 2;
const int Chanel_B = 3;
const int PSOMember = 10;
int Iteration = 3;
float Position_Angle_1[PSOMember];
float Position_Angle_2[PSOMember];
float Velocity_Angle_1[PSOMember];
float Velocity_Angle_2[PSOMember];
float P_best_Angle_1[PSOMember];
float P_best_Angle_2[PSOMember];
float P_best_Value[PSOMember];
float G_best_Angle_1;
float G_best_Angle_2;
float G_best_Value = 0;
float C1 = 1, C2 = 1, C3 = 2;
float Angle_1,Angle_2;
int Delta_Distance;
void setup() {
lcd.begin(16, 2);
Serial.begin(9600);
//----------Encoder----------
attachInterrupt(0, PulseCounter, FALLING);
pinMode(Chanel_A, INPUT_PULLUP);
pinMode(Chanel_B, INPUT_PULLUP);
PulseNumber = 0;
//----------servo-------
Servo_1.attach(PinServo_1);
Servo_2.attach(PinServo_2);
Servo_1.write(180 - 0);
Servo_2.write(0);
//----create fisrt random angle 1&2
for (int i=0; i<=(PSOMember - 1); i++) {

Position_Angle_1[i] = random(0,160);
Position_Angle_2[i] = random(20,100);
Velocity_Angle_1[i] = 0;
Velocity_Angle_2[i] = 0;
P_best_Value[i]= {0};
}
}
void PulseCounter() {
if (digitalRead(Chanel_B)== HIGH)
{PulseNumber++;}
else
{PulseNumber--;}
58
}
void loop() {
PSO();
while (1==1){
lcd.setCursor(0,0);
lcd.print("Gb_1=");
lcd.print(G_best_Angle_1);
lcd.print(PulseNumber);
lcd.setCursor(0,2);
lcd.print("Gb_2=");
lcd.print(G_best_Angle_2);
PulseNumber = 0;
Servo_1.write(180- G_best_Angle_1 );
delay(500);
Servo_2.write(G_best_Angle_2);
delay(500);
Servo_1.write(180-0);
delay(500);
Servo_2.write(30);
delay(500);
}
}
void PSO(){
for (int iter = 0; iter <= Iteration; iter++) {
lcd.setCursor(6,1);
lcd.print("it=");
lcd.println(iter);
for (int i=0; i<=(PSOMember-1); i++) {
Serial.print(" i = ");
Serial.println(i);
Angle_1 = Position_Angle_1[i];
Angle_2 = Position_Angle_2[i];
lcd.setCursor(0, 0);
lcd.print("i=");
lcd.print(i);
//lcd.setCursor(7, 0);
//lcd.print("A1=");
//lcd.print(Angle_1);
//lcd.setCursor(7, 2);
// lcd.print("A2=");
//lcd.print(Angle_2);
Serial.print(" Angle_1 = ");

Serial.print(Angle_1);
Serial.print(" Angle_2 = ");
Serial.println(Angle_2);
Servo_1.write(180 - Angle_1);
delay(500);
Servo_2.write(Angle_2);
delay(500);
Servo_1.write(180-0);
delay(500);
Servo_2.write(0);
delay(500);
//Serial.print( " d = " );

//Serial.print(PulseNumber);
Delta_Distance = PulseNumber;
59
PulseNumber = 0;
lcd.print("d=");
lcd.print(Delta_Distance);
if (Delta_Distance > P_best_Value[i]) {

P_best_Angle_1[i] = Position_Angle_1[i];
P_best_Angle_2[i] = Position_Angle_2[i];
P_best_Value[i] = Delta_Distance;
}
if (Delta_Distance > G_best_Value) {

G_best_Angle_1 = Position_Angle_1[i];;
G_best_Angle_2 = Position_Angle_2[i];;
G_best_Value = Delta_Distance;
}
lcd.print("gb=");
lcd.print(G_best_Value);
Serial.print( " G_best_Angle_1 = ");

Serial.print(G_best_Angle_1);
Serial.print( " G_best_Angle_2 = ");
Serial.print(G_best_Angle_2);
Serial.print( " G_best_Value= ");
Serial.println(G_best_Value);
Serial.print( " P_best_Angle_1 = ");
Serial.print(P_best_Angle_1[i]);
Serial.print( " P_best_Angle_2 = ");
Serial.print(P_best_Angle_2[i]);
Serial.print( " P_best_Value= ");
Serial.println(P_best_Value[i]);
}
for (int i=0; i<=(PSOMember-1); i++) {
Velocity_Angle_1[i] = C1*Velocity_Angle_1[i] +
C2*((float)random(1,1000)/1000)*(P_best_Angle_1[i] - Position_Angle_1[i]) +
C3*((float)random(1,1000)/1000)*(G_best_Angle_1 - Position_Angle_1[i]);
Velocity_Angle_2[i] = C1*Velocity_Angle_2[i] +
C2*((float)random(1,1000)/1000)*(P_best_Angle_2[i] - Position_Angle_2[i]) +
C3*((float)random(1,1000)/1000)*(G_best_Angle_2 - Position_Angle_2[i]);
Position_Angle_1[i] = constrain(Position_Angle_1[i] + Velocity_Angle_1[i], 0,

160);
Position_Angle_2[i] = constrain(Position_Angle_2[i] + Velocity_Angle_2[i], 0,

100);
}
}
}
60
APENDIX OF PSO-PID
CODE
61
RESULTS
iter= 0,member= 0,ki= 0.005000,kp= 0.008000,ki_P_best= 0.0050,kp_P_best= 0.0080,value_P_best= 9786.00,RMSE= 9786.00
ki_G_best= 0.0070, kp_G_best= 0.0090, value_G_best= 7125.00
.
.
iter= 4,member= 2,ki= 0.010996,kp= 0.010658, ki_P_best= 0.0110,kp_P_best= 0.0107,value_P_best= 5646.00,RMSE= 5646.00
ki_best = 0.010996
kp_best = 0.010658
62
63
.
64
ki_best = 0.016003
kp_best = 0.026379
ki_best = 0.016003
kp_best = 0.026379
65
iter= 0,member= 3,ki= 0.0070,kp= 0.009000,ki_P_best= 0.0070, kp_P_best= 0.0090, value_P_best= 7163.00, RMSE=
7163.00
iter= 0,member= 4,ki= 0.0090,kp= 0.002000,ki_P_best= 0.0090, kp_P_best= 0.0020, value_P_best= 8495.00, RMSE=
8495.00
66

.
iter= 23,member= 5,ki= 0.0207,kp= 0.038536,ki_P_best= 0.0204,kp_P_best= 0.0385,value_P_best= 2762.00 RMSE= 2846.00
67
68
iter= 24,member= 9,ki= 0.0201,kp= 0.039036,ki_P_best= 0.0194, kp_P_best= 0.0411,value_P_best= 2749.00,RMSE= 2918.00
69
70
ki_best = 0. 0203
kp_best = 0. 0385
ki_best = 0. 0203
kp_best = 0. 0385
71

Nghiên-cứu-khoa-học final-đã chuyển đổi PDF

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Nghiên-cứu-khoa-học final-đã chuyển đổi PDF

Uploaded by

Copyright:

Available Formats

ĐẠI HỌC THÁI NGUYÊN

TRƯỜNG ĐẠI HỌC KỸ THUẬT CÔNG NGHIỆP

BÁO CÁO TỔNG KẾT

NGHIÊN CỨU CÀI ĐẶT THUẬT TOÁN TỰ HỌC ỨNG DỤNG

Chủ nhiệm đề tài: Nguyễn Văn Hậu

Thái Nguyên- 2018

FACULTY OF INTERNATIONAL TRAINING

RESEARCH AND INSTALLATION OF LEARNING ALGORITHM

Project Leader: Nguyen Van Hau

Thai Nguyen- 2018

Chủ nhiệm đề tài

(chữ ký , học tên)

Thái Nguyên- 2018

Figure 4. 1 Robot model .............................................................................................. 40

Artificial intelligence is an area which is continuously evolving and improving.

The Learning algorithm, which is a form of reinforcement learning, was implemented on

2. Mục Tiêu Đề Tài

3. Tổng Quan Tài Liệu

3.1 Tình hình nghiên cứu ngoài nước

4. Bố Cục Báo Cáo

Chương 2: Tìm hiểu chuyên sâu về thuật toán PSO.

Chương 5: Kết luận

A learning algorithm is a mathematical framework or procedure that calculates the best

The fundamental task in robotics is to create an intelligent machine which achieves a

1.2 Artificial intelligence techniques

Artificial intelligence techniques try to simulate human behavior. They present

1.3 Several Learning Algorithms

1.3.1 Genetic Algorithm (GA)

GA uses the method of natural selection and genetics for obtaining an

1.3.2 Differential Evolution

Differential Evolution was introduced by Storn and Price in 1995. DE is similar to GA

1.3.3 Ant Colony Optimization

Ant Colony Optimization (ACO) was introduced in 1992 by Dorigo. ACO is

1.3.4 Tabu Search

Particle swarm optimization (PSO) is a population based stochastic optimization

PSO shares many similarities with evolutionary computation techniques such as

The algorithm keeps track of three global variables:

2.2 The algorithm

For each particle p in P do

v = v + c1*rand*(pBest – p) + c2*rand*(gBest – p);

c1: weight of local information

c2: weight of global information

pBest: best position of the particle

gBest: best position of the swarm

rand: random variable

vi (t +1) = vi (t) + c1r1[xi (t) − xi (t)] + c2 r2 [g(t) − xi (t)]

✓ xi (t) is the particle‟s position at time t

xi (t +1) = xi (t) + vi (t +1)

A PID Controller is a common technique used to control a wide variety of machines

Figure 3. 1 A typical PID control structure

✓ Kp is the proportional gain, a tuning parameter

Figure 3. 2 Response of PV to step change of SP vs time, for three values of Kp

Figure 3. 3 Response of PV to step change of SP vs time, for three values of Ki

Figure 3. 4 Response of PV to step change of SP vs time, for three values of Kd

Figure 3. 5 Effects of increasing a parameter independently

Figure 3. 7 Calculation for sum of error

Figure 3. 8 Block diagram of system

• Block diagram Description

✓ Case Size: 111 x 78 x 36mm

Figure 3. 9 15V, 3 Amp Single

Figure 3. 10 Arduino MEGA 2560

Microcontroller ATmega 2560

Figure 3. 11 Technical specification of Arduino MEGA 2560

v = v + c1rand(pBest – p) + c2rand(gBest – p);