PRESENTATION

ON
KNOWLEDGE EXTRACTION FROM
NUMERICAL DATA: AN ARTIFICIAL BEE
COLONY BASED APPROACH
By:
Lalit Kumar
Roll No. 96446582903

Under the Supervision of

Dr. Dheerendra Singh
Professor & Head CSE
SUSCET Tangori

Department of Computer Science and Engineering

SUS College of Engineering and Technology
Tangori, Mohali 1
 OUTLINES OF PRESENTATION
INTRODUCTION

LITERATURE REVIEW

PROBLEM FORMULATION

PRESENT WORK

RESULTS

CONCLUSION

FUTURE SCOPE

References

Publications

2
 INTRODUCTION TO FUZZY LOGIC

Two types of sets:

 Crisp Set: Defined in terms of 0 (False) and 1 (True).
 Fuzzy Set: It uses the whole interval between 1 (True)
and 0 (False)

The word fuzzy means “Vagueness”. Fuzziness occurs

when the boundary of a piece of information is not
clear-cut.

3
 FUZZY SYSTEM

Figure 1: Block Diagram of Fuzzy Logic Based System

4
 FUZZY SYSTEM
Fuzzy systems are a class of systems belonging to
knowledge based systems. In fuzzy systems, the
knowledge is represented in the form of a rulebase of
the system. Fuzzy system can be represented with the
help of block diagram as shown in figure 1.

FUZZIFICATON MODULE:
Fuzzification is the process of transforming the crisp
input values to the corresponding values in fuzzy
domain (fuzzy values).

5
 FUZZY SYSTEM

KNOWLEDGE BASE:
 This module contains the knowledge of the
application domain and the procedural knowledge. It
consists of a data base and linguistic control rule
base.
 The data base provides the necessary definitions
which are used to define linguistic control rules and
fuzzy data manipulation in an FLC.

6
 FUZZY SYSTEM
INFERENCE ENGINE:
This module simulates the decision making capabilities
of human brain. It involves three steps:
 Rule Composition: Generally the antecedent of a rule
has more than one part and hence the fuzzy operators
like t-norms, s- norms are applied to obtain the result.
 Implication: The shaping of the consequent based on
the composed value of antecedent is termed as
implication.
 Aggregation: It is a process by which several fuzzy sets
are combined into a desirable way to produce a single
fuzzy set.
7
 FUZZY SYSTEM
DEFUZZIFICATION:
Defuzzification performs the reverse operation of
fuzzification process that is it converts the fuzzified
output of inference engine into corresponding crisp
values.

8
 RULE BASE GENERATION
The rule base for a fuzzy system can be
generated in two ways:
1. Knowledge Based Rule Base.
2. Data Driven Rule Base.

9
KNOWLEDGE BASED RULE BASE
In the knowledge driven fuzzy models, the rule base is
generated by domain experts and knowledge
engineers. Knowledge based approach is significant
for less complex systems.

10
 DATA DRIVEN RULE BASE

In the data driven models, the rule base is generated

from the available numerical data. As, the domain
experts are difficult to find and the requisite
knowledge extraction from the experts itself is a
difficult task, the data driven modeling assumes
significance. For complex systems Data driven
approach is used.

11
 ARTIFICIAL BEE COLONY ALGORITHM
 Artificial Bee Colony (ABC) is one of the most
recently defined algorithms by Dervis karaboga in
2005, motivated by the intelligent behaviour of honey
bees.
 ABC is developed based on inspecting the behaviors
of real bees on finding nectar and sharing the
information of food sources to the bees in the hive
 The bees have their own type of behavior that they
always follow and that are in the food sources is that
the bees has all it to nest, they have a lot of target or
aim at the energy departments. They have a lot of easy
tricks to gain a great amount of energy too.
12
 Artificial Bee Colony (ABC)
 Swarm Intelligence employs the collective
behaviors in the animal societies to design
algorithms.

 ABC is developed based on inspecting the

behaviors of real bees on finding nectar and
sharing the information of food sources to the
bees in the hive.

 Agents in ABC:
◦ The Employed Bee
◦ The Onlooker Bee
◦ The Scout
13
Artificial Bee Colony Algorithm

14
 Artificial Bee Colony (ABC)
The Employed Bee:
It stays on a food source and provides the
neighborhood of the source in its memory.

The Onlooker Bee:

It gets the information of food sources from
the employed bees in the hive and select
one of the food source to gathers the nectar.

The Scout:
It is responsible for finding new food, the
new nectar, sources.
15
LITERATURE REVIEW

 Singh D. (2012) proposed Real Optimization Problem in the paper “Solving Real
Optimization Problem using Genetic Algorithm with Employed Bee” using Genetic
algorithm with Employed Bee (GAEB) multimodal function has two or more local
optima. A function of variables is separable if it can be rewritten as a sum of
functions of just one variable. The search process for a multimodal function is
difficult if the local optima are randomly distributed.
 Ashita S. Bhagade(2012) is presented a paper “Artificial Bee Colony (ABC)
Algorithm for Vehicle Routing Optimization Problem” by considering Artificial Bee
Colony approach. The Artificial Bee Colony (ABC) Algorithm can be used to solve
several optimal problems. The algorithm is aimed to minimize the length of the tour
and find the optimal path. To obtain performance comparisons with the other
method, simulation framework is developed.
 Manish Gupta et al. (2012) proposed “An Efficient Modified Artificial Bee Colony
Algorithm for Job Scheduling Problem” which explained real coded mutation. The
crossover operator is applied to the ABC after the employed bee phase and onlooker
bee phase of ABC algorithm. The algorithm researches some probabilistic criteria
selected food source is altered by mutation operator. The experiments are performed
on a job scheduling problem available in the literature.

16
LITERATURE REVIEW …
 Mustafa M. Noaman et al. (2011) proposed the Artificial Bee Colony
(ABC) Algorithm in the paper “Artificial Bee Colony based Data Mining
Algorithms for Classification Tasks”. In the paper, the Shortest Common
Super sequence problem has been solved and the results obtained are
compared by applying the Artificial Bee Colony (ABC) with the results
obtained from applying other approaches that were proposed for solving the
SCSP.

 Malek Alzaqebah et al. (2011) has compared the performance of the ABC
algorithm by different selection strategies in the paper “Artificial bee
colony search algorithm for examination timetabling Problems”. ABC
algorithm with a disruptive selection strategy is able to produce better
results when compared to other selection strategies tested in this work. We
believe the performance of the ABC algorithm can be enhanced by
applying a suitable mechanism to choose the neighborhood structure based
on the current solution in hand

17
LITERATURE REVIEW …
 Ivona B. et al. (2010) presented the ACO algorithm for capacitated vehicle routing
problem in the paper “Ant colony system: a cooperative learning approach to the
travelling salesman problem”. The twelve benchmark instances of small scale
problems were tested. The results were compared to the best known results.
Although the global optimality cannot be guaranteed, the performance of the
algorithm is good and robust. It is noticed that algorithm can be trapped in the local
minimum for some benchmark instances.

 D. Karaboga (2005) and his research group have researched the ABC algorithm and
its applications to real world problems. Karaboga and Basturk have investigated the
performance of the ABC algorithm on unconstrained numerical optimization
problems and its extended version for the constrained optimization problems and
Karaboga et al. applied ABC algorithm to neural network training

 Zadeh A. (1965) outlined the principles of fuzzy modeling in the paper . He gave the
concept of grade of membership. The significance of this paper was that it
challenged not only probability theory as the sole agent for uncertainty but the very
foundations on which probability theory was based:Aristotelian two-valued logic

18
 ALGORITHM
Algorithm to find Travelling Salesman Problem :

 The main steps of the algorithm are given below:

 Initial food sources are produced for all employed bees
◦ REPEAT
 Each employed bee goes to a food source in her memory and determines a
neighbor source, then evaluates its nectar amount and dances in the hive
 Each onlooker watches the dance of employed bees and chooses one of their
sources depending on the dances, and then goes to that source. After
choosing a neighbor around that, she evaluates its nectar amount.
 Abandoned food sources are determined and are replaced with the new food
sources discovered by scouts.
 The best food source found so far is registered.
 UNTIL (requirements are met)

19
FORMULATION OF PROBLEM

Figure 5: Sugeno Type Fuzzy System

20
 FORMULATION OF PROBLEM
 For sugeno type system:

Ocomputed=(W1*C1+W2*C2+...+Wn*Cn)/ (W1+W2+….+Wn) (1)

 Compare this computed output with actual output as given in

the data set and find the error. Let the error be defined as:
E= Actual output (As given in data set) – Computed output (As
given in equation 1) (2)
 Now the whole problem of rulebase generation boils down to
a minimization problem as stated below:
Minimize objective function E
E = Oactual – Ocomputed (3)
21
 METHODOLOGY/ PROPOSED
WORK
 In order to solve the minimization problem using ABC the
problem has to be converted to a weighted graph.
 Number of nodes = n+1 where n=number of rules.
 Each path represents one rule base, so number of paths equal to
number of possible rule-bases.
 Since for each rule, select one appropriate consequent from the
available choices (C1, C2, C3, ….Ck). The network graph has
parallel paths between each pair of nodes equal to the number
of consequents available as shown in fig 5.
 Divide these distances by half and provide intermediate nodes
(fig. 5).

22
OBJECTIVES OF DISSERTATION

 To generate membership functions using clustering

techniques.

 Automatic generation of rule base from given data

using ABC based approach.

 Comparison with other soft computing techniques

(GA, BBO, Neural networks etc.).

23
 METHODOLOGY/PROPOSED WORK
 The consequent has to be chosen in such a way as to
minimize error.
 This error is used to update the pheromone trail i.e.
∆τk. Smaller the error more is the amount of
pheromone that is being deposited on the path as
shown: ∆τk = 1/error (E).
 This allows artificial bees to choose a path with
higher pheromone deposit with higher probability.
 Finally all the bees follow a path that has high
pheromone deposit leading to shortest path i.e. path
with least error. This leads to generation of a rule that
produces minimum error. 24
METHODOLOGY/PROPOSED
WORK
Start

Fuzzification of Inputs

Composition of Inputs (Wi’s)

Call ABC to Choose Appropriate Ci’s

Compute Output using (ΣWi * Ci /Σ Wi)

Evaluate Error of the System (E=OActual -OComputed )

Update Pheromone on the basis of Error (∆τk = 1/ Error)

Generate Rulebase for a given System

Call Rule Reduction Algorithm

End

Figure 7: Methodology for Rulebase Generation Using ABC 25

 APPLICATIONS OF ABC

 Travelling Salesman Problem (TSP)

26
 Travelling Salesman Problem

The suggested approach has been applied for

identification of fuzzy model for the Travelling
Salesman Problem [31].

 Number of City travels by salesman : 20

 Important route/node cover by salesman : 10
 Profit of using ABC approach : 13766

27
 (RESULTS)

Figure 8: Travelling Salesman Fuzzy Model

28
 SAMPLE RUN OF PROGRAM
AND ITS GENERATED OUTPUT

Fig:-9 Final Extracted Rules

The above rules are the final extracted rules after applying rule reduction technique.
This technique will help to choose an appropriate rule when two rules with same
antecedent but different consequent result. After discarding redundant &
contradicting rules the complete set of rules that constitute the final compact rule
base for the system is obtained. 29
 SIMULATION RESULTS
The results for mentioned problem is in terms of its
Mean Square Error (MSE) which is given by:
MSE= 1/2N Σk=1 [y(k)-y’(k)]2
Where, y(k) = actual output
y’(k) = computed output
N = number of data points taken for model
validation
MSE for the given data set is : MSE= 0.03

30
 SIMULATION RESULTS
The Comparison with Some of the existing approaches
is as shown in Figure 10:
0.14

0.12

0.1

0.08
MSE

0.06 Series1

0.04

0.02

GA Wang ACO ABC

Approaches
31
 SIMULATION RESULTS

100.00%
Classification Rate

98.00%

96.00%

94.00%

92.00%
Series1
90.00%

88.00%

86.00%

GA Wang ACO ABC

Approach
32
 CONCLUSIONS
 Rule extraction from numerical data is a high computational complexity
problem. ABC is applied successfully to extract rule base from numerical data.
 The problem is converted in to weighted graph whose edge length represented
movement for fuzzification and distance equal to (Wi * Ci / Wi) of the
corresponding rule for rulebase generation.
 The difference between computed output (Wi * Ci / Wi) and actual output as
given in the training example gives error. This error is used to update
pheromone trail.
 Smaller the error more is the amount of pheromone being deposited on the
path.
 This allows artificial bees to choose a path with higher pheromone with higher
probability. Finally all the ants follow a path that has high pheromone strength
leading to shortest path i.e. least error. This leads to optimized membership
functions and generation of rules that produces minimum error.
 The proposed approach is successfully applied on Travelling Salesman
problem with a Mean Square Error of 0.03 and classification rate of 98.20%.
The optimized rules are extracted from given data set for sugeno type system.
The method appears to be efficient.
33
 FUTURE SCOPE
Fuzzy rule based systems are a class of knowledge based
systems. The intelligence of a fuzzy logic based system lies in
its rule base. This dissertation work is based on the generation
of rulebase automatically from the numerical data as the
availability of domain experts is not possible for more complex
systems.
The present work is focused on rulebase extraction only
however the same approach must be applied on complete
system identification. In that the process involved input –
output membership function generation, tuning along with
rulebase generation.
There are some other soft computing techniques like BBO, BB-
BC Optimization that can also be applied on the proposed work
and their performance can be compared with the proposed
approach. 34
 REFERENCES
 [1] Adil B., “Artificial Bee Colony Algorithm and Its Application to Generalized
Assignment Problem” International Conference on Computational Intelligence for Modeling,
Control and Automation, Las Vegas.
 [2] Aruna K., Shakti Kumar, KK Aggarwal, Jagatpreet Singh,”Particle Swarm Optimizer
for fuzzy models IEEE Proc. on Fuzzy Systems, 2007
 [3] Ashita S. Bhagade, “Artificial Bee Colony (ABC) Algorithm for Vehicle Routing
Optimization Problem” International Journal of Soft Computing and Engineering (IJSCE)
ISSN: 2231-2307, Volume-2, Issue-2, May 2012
 [4] Bhalla P., “Fuzzy Rule base generation from Numerical Data using Ant colony
optimization,” MAIMT-Journal of IT & Management. Vol. 1, No. 1 May-Oct, 2007, pp 33-47.
 [5] Chang J., Zhao Yongsheng, Wen Chongzhu, “Research on Optimization of Fuzzy
Membership function based on Ant Colony Algorithm,” Proc of the 25th Chinese Control
Conference, Harbin, Aug, 2006.
 [6] Chia-Feng J, H.J. Huang and C.M. Lu, “Fuzzy Controller Design by ant colony
optimization,” IEEE Proc. on Fuzzy Systems, 2007.
 [7] D.G., Genetic Algorithm in Search. Optimization and Machine Learning, Addison-
Wesley Publishing Company, USA, 1989.

35
 REFERENCES
 [8] D. Karaboga and B. Basturk, On the performance of artificial bee colony (ABC)
algorithm, Applied Soft Computing 8 (2008), pp.687-697, 2008.
 [9] D. Karaboga, An Idea Based On Honey Bee Swarm For Numerical Optimization,
Technical Report-TR06, Erciyes University, Engineering Faculty, Computer
Engineering Department, 2005.
 [10] E. Bonabeau, M. Dorigo, and G. Theraulaz, Swarm Intelligence: From Natural to
Artificial Intelligence, NY: Oxford University Press, 1999.
 [11] G. A. Gurzadyan, Space Dynamics, Taylor & Francis Inc, London, 2002.
 [12] Gupta M., “An Efficient Modified Artificial Bee Colony Algorithm for Job
Scheduling Problem” International Journal of Soft Computing and Engineering
(IJSCE) ISSN: 2231-2307, Volume-1, Issue-6, January 2012
 [13] Inova B., “Artificial bee colony algorithm for the capacitated vehicle routing
problem” Proceedings of the European Computing Conference 2010.
 [14] J.-F. Chang, S.-C. Chu, J. F. Roddick, and J.-S. Pan, A parallel particle swarm
optimization algorithm with communication strategies, Journal of Information
Science and Engineering, vol.21, no.4, pp.809-818, 2005.
 [15] Kumar S. “Introduction to Fuzzy Logic Based Systems”, Workshop on Intelligent
System Engineering (WISE-2010), 2010.

36
 REFERENCES
 [16] M. Dorigo and L. M. Gambardella, Ant colony system: a cooperative learning
approach to the traveling salesman problem, IEEE Trans. on Evolutionary
Computation, vol.26, no.1, pp.53-66, 1997. 12 PEI-WEI TSAI, JENG-SHYANG PAN,
BIN-YIH LIAO, AND SHU-CHUAN CHU
 [17] M. Galea and Q. Shen, “Fuzzy Rules from ant-inspired computation,”Proc. IEEE Int
Fuzzy Systems, pp 1691-1696, 2004.
 [18] M.S. Abadeh, J. Habibi and E. Soroush, “Induction of Fuzzy classification systems
using evolutionary ABC-based algorithms,” Proc. of the First Asia Int’l Conf. on
Modeling and Simulation (AMS’07), 2007
 [19] Malek Alzaqebah, “Artificial bee colony search algorithm for examination timetabling
Problems” International Journal of the Physical Sciences Vol. 6(17), pp. 4264-4272,
September, 2011
 [20] Malek Alzaqebah, “Comparison on the Selection Strategies in the Artificial Bee
Colony Algorithm for Examination Timetabling Problems” International Journal of
Soft Computing and Engineering (IJSCE) ISSN: 2231-2307, 2011
 [21] Marco Dorigo and Thomas Stuzzle, Ant Colony Optimization, Eastern Economy
Edition, PHI, 2005.
 [22] Marco Dorigo and Thomas Stuzzle, Ant Colony Optimization, Eastern Economy
Edition, PHI, 2005.
 [23] Mohd Afizi Mohd Shukran, “Artificial Bee Colony based Data Mining Algorithms for
37
Classification Tasks” 2011.
 REFERENCES
 [24] Mustafa M. Noaman, “Solving Shortest Common Supersequence Problem Using
Artificial Bee Colony Algorithm” The Research Bulletin of Jordan ACM, ISSN,
Volume II (III) PP-80.
 [25] N. Iwasaki and K. Yasuda, Adaptive particle swarm optimization using velocity
feedback, International Journal of Innovative Computing, Information and Control,
vol.1, no.3, pp.369-380, 2005.
 [26] S.-C. Chu and P.-W. Tsai, Computational intelligence based on the behavior of cats,
International Journal of Innovative Computing, Information and Control, vol.3,
no.1, pp.163-173, 2007.
 [27] S.-C. Chu, J. F. Roddick, and J.-S. Pan, Ant colony system with communication
strategies, Information Sciences, vol.167 (2004), pp.63-76, 2004.
 [28] S.-C. Chu, J. F. Roddick, C.-J. Su, and J.-S. Pan, Constrained Ant Colony
Optimization for Data Clustering, Proc. of 8th Pacific Rim International Conference
on Artificial Intelligence, Auckland, New Zealand, LNAI 3157, pp. 534-543, 2004.
 [29] S.-C. Chu, P.-W. Tsai, and J.-S. Pan, Cat Swarm Optimization, Proc. of 9th Pacific
Rim International Conference on Artificial Intelligence, Guilin, China, LNCS 4099,
pp.854-858, 2006.
 [30] Shakti K, P. Bhalla and S.Sharma, “Automatic Fuzzy Rule base Generation for
Intersystem Handover using Ant Colony Optimization Algorithm,” International
Conference on Intelligent Systems and Networks (IISN-2007), Feb 23-25, 2007,
MAIMT, Jagadhri Haryana, India, pp 764-773.

38
 REFERENCES
 [31] Shakti Kumar, “Rule base generation using ant colony optimization,” Proc. Of the
one week workshop on applied soft computing (SOCO-2006), Haryana Engineering
College, Jagadhri, July 2006.
 [32] Shakti Kumar, P.Bhalla and Amarpartap Singh, “Soft Computing Approaches to
Fuzzy System identification:A Survey”, IISN-2009,pp 402-411, 2009.
 [33] Singh D., “Solving Real Optimization Problem using Genetic Algorithm with
Employed Bee” International Journal of Computer Applications (0975 – 8887)
Volume 42– No.11, March 2012.
 [34] X. Cai, Z. Cui, J. Zeng, and Y. Tan, Particle Swarm Optimization with Self-
adjusting Cognitive Selection Strategy, International Journal of Innovative
Computing,information and Control, vol.4, pp.943-952, 2008.
 [35] X.-Z. Gao, S. J. Ovaska, and X. Wang, A GA-based Negative Selection Algorithm,
Internationa Journal of Innovative Computing, Information and Control, vol.4, no.4,
pp.971-979, 2008.
 [36] Y. Guo, X. Gao, H. Yin, and Z. Tang, Coevolutionary Optimization Algorithm with
Dynamic Subpopulation Size, International Journal of Innovative Computing,
Information and Control, vol.3, no.2, pp.435-448, 2007.
 [37] Z. Cui, J. Zeng, and G. Sun, A Fast Particle Swarm Optimization, International
Journal of Innovative Computing, Information and Control, vol.2, no.6, pp.1365-
1380, 2006.
 39
 PUBLICATIONS
 Paper entitled “Solving NP Hard Problem Using Artificial
Bee Colony Algorithm” published in International Journal of
Computer Engineering Technology (IAEME-IJCET) ISSN
No. 0976-6367 Volume 4, Issue 1, January-February (2013),
pp. 171-177.

 Paper entitled “Knowledge Extraction from Numerical Data:
An ABC Based Approach” published in International Journal
of Computer Engineering Technology (IAEME-IJCET) ISSN
No. 0976-6367. Volume 4, Issue 2, March-April (2013), pp.
01-09.

40
THANKYOU

41