Professional Documents
Culture Documents
Neuro Fuzzy and Genetic Programming Unit Iii
Neuro Fuzzy and Genetic Programming Unit Iii
Neuro Fuzzy and Genetic Programming Unit Iii
Programming
UNIT III
Unit 3 Syllabus
Duration 9 9 9 9 9
(hour)
History of Evolutionary
SLO-1 Biological and Artificial Delta Rule, Derivation of GDR Crisp sets Fuzzification of Input Computing,
S-1 Neuron Variables, Application Genetic Algorithms, basic
of Fuzzy operations concepts
SLO-2 History of ANN Backpropagation Algorithm, Local Fuzzy sets GA Cycle , Fitness Function,
Minima Problem
SLO-1 ANN architectures Radial Basis Function Neural Network Fuzzy membership functions Introduction to GA Operators
Evaluation of Fuzzy rules, Selection Operators,
S-2 SLO-2 Learning Algorithms Pattern Association, Auto Associative Operations of Fuzzy sets Aggregation of output Fuzzy
sets Crossover, Mutation
nets Operations
Activation Functions, Rule based systems,
SLO-1 Bias, Threshold and Hetero Associative nets Fuzzy Relations, Operations Conventional programs
S-3 Schema Theorem, Example
other parameters vs Rule based systems
SLO-2 McCulloch Pitts model, Bidirectional Associative Memory Fuzzy Extension Principle Fuzzy Propositions
Network
SLO-1 Crisp Relations, Fuzzy relations,
S-4 SLO-2 Simulation of Logic Hopfield network Competitive networks: Properties, operations, Fuzzification Classification of Genetic
Functions Maxnet Algorithm
SLO-1 Perceptron Network Self Organizing Map Network Propositional Logic Defuzzification Holland Classifier Systems
S-5 Learning Vector Quantization Fuzzy Controller : Air
SLO-2 Hebbian network Crisp Logic conditioner control, Genetic Programming
Cruise Controller
SLO-1 ADALINE networks Predicate Logic Rules of Data Representation
S-6 Adaptive Resonance Theory Network Inference Fuzzy Decision making
SLO-2 MADALINE networks Fuzzy Truth, Fuzzy Rules Genetic Operators
Introduction to neuro fuzzy
SLO-1 Practice of Neural Network tool : Delta Fuzzy Reasoning system- Adaptive Neuro- Application of Genetic
S- Practice of Neural rule Fuzzy Inference Systems Algorithm
Network tool : Simple Coactive Neuro-Fuzzy
7,8 Modeling
Logic functions
Practice of Neural Network
SLO-2 tool : Pattern Classification Practice of Fuzzy Logic tool: Recent Applications
Practice of Optimization
Fuzzy functions and Genetic algorithm tool
SLO-1 Practice of Neural Practice of Fuzzy Logic Practice of Fuzzy Logic
S-9 SLO-2 Network tool : XOR Practice of Neural Network tool : tool: Fuzzy operations tool : Fuzzy controller
problem Pattern Clustering design and applications
NFGP- Unit 1 2
Introduction
Complement of set A.
AB=BA
AB=BA
A (B C) = (A B) C
A (B C) = (A B) C
A (B C) = (A B) (A C)
A (B C) = (A B) (A C)
AA=A
AA=A
AX=X
AX=A
A=A
A=
Mapping of Classical Sets to Functions
Mapping is an important concept in relating set-theoretic forms to function-
theoretic representations of information. In its most general form it can be used to
map elements or subsets in one universe of discourse to elements or sets in
another universe.
Unit 3 Syllabus
Duration 9 9 9 9 9
(hour)
History of Evolutionary
SLO-1 Biological and Artificial Delta Rule, Derivation of GDR Crisp sets Fuzzification of Input Computing,
S-1 Neuron Variables, Application Genetic Algorithms, basic
of Fuzzy operations concepts
SLO-2 History of ANN Backpropagation Algorithm, Local Fuzzy sets GA Cycle , Fitness Function,
Minima Problem
SLO-1 ANN architectures Radial Basis Function Neural Network Fuzzy membership functions Introduction to GA Operators
Evaluation of Fuzzy rules, Selection Operators,
S-2 SLO-2 Learning Algorithms Pattern Association, Auto Associative Operations of Fuzzy sets Aggregation of output Fuzzy
sets Crossover, Mutation
nets Operations
Activation Functions, Rule based systems,
SLO-1 Bias, Threshold and Hetero Associative nets Fuzzy Relations, Operations Conventional programs
S-3 Schema Theorem, Example
other parameters vs Rule based systems
SLO-2 McCulloch Pitts model, Bidirectional Associative Memory Fuzzy Extension Principle Fuzzy Propositions
Network
SLO-1 Crisp Relations, Fuzzy relations,
S-4 SLO-2 Simulation of Logic Hopfield network Competitive networks: Properties, operations, Fuzzification Classification of Genetic
Functions Maxnet Algorithm
SLO-1 Perceptron Network Self Organizing Map Network Propositional Logic Defuzzification Holland Classifier Systems
S-5 Learning Vector Quantization Fuzzy Controller : Air
SLO-2 Hebbian network Crisp Logic conditioner control, Genetic Programming
Cruise Controller
SLO-1 ADALINE networks Predicate Logic Rules of Data Representation
S-6 Adaptive Resonance Theory Network Inference Fuzzy Decision making
SLO-2 MADALINE networks Fuzzy Truth, Fuzzy Rules Genetic Operators
Introduction to neuro fuzzy
SLO-1 Practice of Neural Network tool : Delta Fuzzy Reasoning system- Adaptive Neuro- Application of Genetic
S- Practice of Neural rule Fuzzy Inference Systems Algorithm
Network tool : Simple Coactive Neuro-Fuzzy
7,8 Modeling
Logic functions
Practice of Neural Network
SLO-2 tool : Pattern Classification Practice of Fuzzy Logic tool: Recent Applications
Practice of Optimization
Fuzzy functions and Genetic algorithm tool
SLO-1 Practice of Neural Practice of Fuzzy Logic Practice of Fuzzy Logic
S-9 SLO-2 Network tool : XOR Practice of Neural Network tool : tool: Fuzzy operations tool : Fuzzy controller
problem Pattern Clustering design and applications
NFGP- Unit 1 22
Fuzzy Sets
Fuzzy Sets
A fuzzy set, is a set containing elements that have varying degrees of
membership in the set.
function
membership
U : universe of
discourse.
Fuzzy Sets
Characteristic function X, indicating the belongingness
of x to the set A
X(x) = 1 x A
0 xA
or called membership
Hence,
A B XA B(x)
= XA(x) XB(x)
= max(XA(x),XB(x))
Note: Some books use + for , but still it is not ordinary
addition!
FuzzyFuzzy
SetsSets
A B XA B(x)
= XA(x) XB(x)
= min(XA(x),XB(x))
A’ XA’(x)
= 1 – XA(x)
A’’ = A
Unit 3 Syllabus
Duration 9 9 9 9 9
(hour)
History of Evolutionary
SLO-1 Biological and Artificial Delta Rule, Derivation of GDR Crisp sets Fuzzification of Input Computing,
S-1 Neuron Variables, Application Genetic Algorithms, basic
of Fuzzy operations concepts
SLO-2 History of ANN Backpropagation Algorithm, Local Fuzzy sets GA Cycle , Fitness Function,
Minima Problem
SLO-1 ANN architectures Radial Basis Function Neural Network Fuzzy membership functions Introduction to GA Operators
Evaluation of Fuzzy rules, Selection Operators,
S-2 SLO-2 Learning Algorithms Pattern Association, Auto Associative Operations of Fuzzy sets Aggregation of output Fuzzy
sets Crossover, Mutation
nets Operations
Activation Functions, Rule based systems,
SLO-1 Bias, Threshold and Hetero Associative nets Fuzzy Relations, Operations Conventional programs
S-3 Schema Theorem, Example
other parameters vs Rule based systems
SLO-2 McCulloch Pitts model, Bidirectional Associative Memory Fuzzy Extension Principle Fuzzy Propositions
Network
SLO-1 Crisp Relations, Fuzzy relations,
S-4 SLO-2 Simulation of Logic Hopfield network Competitive networks: Properties, operations, Fuzzification Classification of Genetic
Functions Maxnet Algorithm
SLO-1 Perceptron Network Self Organizing Map Network Propositional Logic Defuzzification Holland Classifier Systems
S-5 Learning Vector Quantization Fuzzy Controller : Air
SLO-2 Hebbian network Crisp Logic conditioner control, Genetic Programming
Cruise Controller
SLO-1 ADALINE networks Predicate Logic Rules of Data Representation
S-6 Adaptive Resonance Theory Network Inference Fuzzy Decision making
SLO-2 MADALINE networks Fuzzy Truth, Fuzzy Rules Genetic Operators
Introduction to neuro fuzzy
SLO-1 Practice of Neural Network tool : Delta Fuzzy Reasoning system- Adaptive Neuro- Application of Genetic
S- Practice of Neural rule Fuzzy Inference Systems Algorithm
Network tool : Simple Coactive Neuro-Fuzzy
7,8 Modeling
Logic functions
Practice of Neural Network
SLO-2 tool : Pattern Classification Practice of Fuzzy Logic tool: Recent Applications
Practice of Optimization
Fuzzy functions and Genetic algorithm tool
SLO-1 Practice of Neural Practice of Fuzzy Logic Practice of Fuzzy Logic
S-9 SLO-2 Network tool : XOR Practice of Neural Network tool : tool: Fuzzy operations tool : Fuzzy controller
problem Pattern Clustering design and applications
NFGP- Unit 1 29
Fuzzy Set Operations
A B(x) = A(x) B(x)
= max(A(x), B(x))
A B(x) = A(x) B(x)
= min(A(x), B(x))
A’(x) = 1 - A(x)
.
Operations
A B
AB AB A
Examples of Fuzzy Set Operations
• Fuzzy union (): the union of two fuzzy sets is the
maximum (MAX) of each element from two sets.
• E.g.
• A = {1.0, 0.20, 0.75}
• B = {0.2, 0.45, 0.50}
• A B = {MAX(1.0, 0.2), MAX(0.20, 0.45), MAX(0.75, 0.50)} =
{1.0, 0.45, 0.75}
Examples of Fuzzy Set Operations
• Fuzzy intersection (): the intersection of two fuzzy sets is just the
MIN of each element from the two sets.
• E.g.
• A B = {MIN(1.0, 0.2), MIN(0.20, 0.45), MIN(0.75, 0.50)} = {0.2, 0.20, 0.50}
Examples of Fuzzy Set Operations
•A = {1/a, 0.3/b, 0.2/c 0.8/d, 0/e}
B = {0.6/a, 0.9/b, 0.1/c, 0.3/d, 0.2/e}
Complement:
= {0/a, 0.7/b, 0.8/c 0.2/d, 1/e}
Union:
A B = {1/a, 0.9/b, 0.2/c, 0.8/d, 0.2/e}
Intersection:
A B = {0.6/a, 0.3/b, 0.1/c, 0.3/d, 0/e}
Properties of Fuzzy Sets
AB=BA
AB=BA
A (B C) = (A B) C
A (B C) = (A B) C
A (B C) = (A B) (A C)
A (B C) = (A B) (A C)
AA=A AA=A
AX=X AX=A
A=A A=
If A B C, then A C
A’’ = A
Fuzzy Sets
Note (x) [0,1]
not {0,1} like Crisp set
A = {A(x1) / x1 + A(x2) / x2 + …}
= { A(xi) / xi}
Note: ‘+’ add
‘/ ’ divide
Only for representing element and its membership.
Also some books use (x) for Crisp Sets too.
Example (Discrete Universe)
A ( x)
0.5
0
2 4 6 8
x : # courses
Example (Discrete Universe)
Alternative Representation:
B ( x, B ( x)) x U
B ( x)
1 about 50 years old
4
x 50
1 1.2
5 1
0.8
B ( x) 0.6
0.4
0.2
0
0 20 40 60 80 100
x : age
Alternative Notation
A ( x, A ( x)) x U
U : discrete universe A
xi U
A ( xi ) / xi
U : continuous universe A A ( x) / x
U
A B
1 1
0.75
0.375
0 0
• (AB = C) (C = 0.75)
Fuzzy Conjunction
• AB min(A, B)
• AB = C "Quality C is the conjunction of Quality A and
B"
A B
1 1
0.75
0.375
0 0
• (AB = C) (C = 0.375)
Example: Fuzzy Conjunction
Calculate AB given that A is .4 and B is 20
A B
1 1
0 0
.1 .2 .3 .4 .5 .6 .7 .8 .9 1 5 10 15 20 25 30 35 40
Example: Fuzzy Conjunction
Calculate AB given that A is .4 and B is 20
A B
1 1
0 0
.1 .2 .3 .4 .5 .6 .7 .8 .9 1 5 10 15 20 25 30 35 40
0.7
0 0
.1 .2 .3 .4 .5 .6 .7 .8 .9 1 5 10 15 20 25 30 35 40
0 0
.1 .2 .3 .4 .5 .6 .7 .8 .9 1 5 10 15 20 25 30 35 40
NFGP- Unit 1 50
Fuzzy Relations
…
Crisp Relation (R)
b1
a1
b2
A a2
a3
b3 B
b4
a4 b5
R A B
Crisp Relation (R)
R A B
b1
a1
b2
A a2
a3
b3 B
b4
a4 b5
1 0 1 0 0 a1 R b1 a1 R b3 a 2 R b5
0 1
0 0 0 ( a1 , b1 ), ( a1 , b3 ), ( a 2 , b5 )
M R
R
1 0 0 1 0 ( a
3 1 , b ), ( a 3 , b 4 ), ( a 4 , b )
2
0 1 0 0 0 a 3 R b1 a 3 R b 4 a 4 R b 2
Crisp Relations
Example:
If X = {1,2,3}
Y = {a,b,c}
R = { (1 a),(1 c),(2 a),(2 b),(3 b),(3 c) }
a b c
1 1 0 1
R= 2 1 1 0
3 0 1 1
Using a diagram to represent the relation
The Real-Life Relation
x is close to y
x and y are numbers
x depends on y
x and y are events
x and y look alike
x and y are persons or objects
If x is large, then y is small
x is an observed reading and y is a
corresponding action
Fuzzy Relations
• Triples showing connection between two sets:
(a,b,#): a is related to b with degree #
…
Fuzzy Relations Matrices
• Example: Color-Ripeness relation for tomatoes
green 1 0.5 0
Red 0 0.2 1
Composition
Let R be a relation that relates, or maps, elements from universe X to
universe Y, and let S be a relation that relates, or maps, elements
from universe Y to universe Z.
yY
Max-Min Composition
X Y Z
R: fuzzy relation defined on X and Y.
y R ( x, y ) S ( y , z )
S R (x, y ) max v min R ( x, v), S (v, y )
Example
R a b c d S
1 0.1 0.2 0.0 1.0 a 0.9 0.0 0.3
2 0.3 0.3 0.0 0.2 b 0.2 1.0 0.8
3 0.8 0.9 1.0 0.4 c 0.8 0.0 0.7
0.1 0.2 0.0 1.0
d 0.4 0.2 0.3
min 0.9 0.2 0.8 0.4
max 0.1 0.2 0.0 0.4
RS
1 0.4 0.2 0.3
2 0.3 0.3 0.3
3 0.8 0.9 0.8
.
Max-Product Composition
X Y Z
R: fuzzy relation defined on X and Y.
R a b c d S
1 0.1 0.2 0.0 1.0 a 0.9 0.0 0.3
2 0.3 0.3 0.0 0.2 b 0.2 1.0 0.8
3 0.8 0.9 1.0 0.4 c 0.8 0.0 0.7
0.1 0.2 0.0 1.0
d 0.4 0.2 0.3
Product 0.9 0.2 0.8 0.4
max .09 .04 0.0 0.4
RS
1 0.4 0.2 0.3
2 0.27 0.3 0.24
3 0.8 0.9 0.7
Unit 3 Syllabus
Duration 9 9 9 9 9
(hour)
History of Evolutionary
SLO-1 Biological and Artificial Delta Rule, Derivation of GDR Crisp sets Fuzzification of Input Computing,
S-1 Neuron Variables, Application Genetic Algorithms, basic
of Fuzzy operations concepts
SLO-2 History of ANN Backpropagation Algorithm, Local Fuzzy sets GA Cycle , Fitness Function,
Minima Problem
SLO-1 ANN architectures Radial Basis Function Neural Network Fuzzy membership functions Introduction to GA Operators
Evaluation of Fuzzy rules, Selection Operators,
S-2 SLO-2 Learning Algorithms Pattern Association, Auto Associative Operations of Fuzzy sets Aggregation of output Fuzzy
sets Crossover, Mutation
nets Operations
Activation Functions, Rule based systems,
SLO-1 Bias, Threshold and Hetero Associative nets Fuzzy Relations, Operations Conventional programs
S-3 Schema Theorem, Example
other parameters vs Rule based systems
SLO-2 McCulloch Pitts model, Bidirectional Associative Memory Fuzzy Extension Principle Fuzzy Propositions
Network
SLO-1 Crisp Relations, Fuzzy relations,
S-4 SLO-2 Simulation of Logic Hopfield network Competitive networks: Properties, operations, Fuzzification Classification of Genetic
Functions Maxnet Algorithm
SLO-1 Perceptron Network Self Organizing Map Network Propositional Logic Defuzzification Holland Classifier Systems
S-5 Learning Vector Quantization Fuzzy Controller : Air
SLO-2 Hebbian network Crisp Logic conditioner control, Genetic Programming
Cruise Controller
SLO-1 ADALINE networks Predicate Logic Rules of Data Representation
S-6 Adaptive Resonance Theory Network Inference Fuzzy Decision making
SLO-2 MADALINE networks Fuzzy Truth, Fuzzy Rules Genetic Operators
Introduction to neuro fuzzy
SLO-1 Practice of Neural Network tool : Delta Fuzzy Reasoning system- Adaptive Neuro- Application of Genetic
S- Practice of Neural rule Fuzzy Inference Systems Algorithm
Network tool : Simple Coactive Neuro-Fuzzy
7,8 Modeling
Logic functions
Practice of Neural Network
SLO-2 tool : Pattern Classification Practice of Fuzzy Logic tool: Recent Applications
Practice of Optimization
Fuzzy functions and Genetic algorithm tool
SLO-1 Practice of Neural Practice of Fuzzy Logic Practice of Fuzzy Logic
S-9 SLO-2 Network tool : XOR Practice of Neural Network tool : tool: Fuzzy operations tool : Fuzzy controller
problem Pattern Clustering design and applications
NFGP- Unit 1 64
Properties of Fuzzy Relations
Example:
y1 y2 z1 z2 z3
R = x1 0.7 0.5 S = y1 0.9 0.6 0.2
x2 0.8 0.4 y2 0.1 0.7 0.5
z1 z2 z3
Using max-min, T = x1 0.7 0.6 0.5
x2 0.8 0.6 0.4
z1 z2 z3
Using max-product, T = x1 0.63 0.42 0.25
x2 0.72 0.48 0.20
Example
NFGP- Unit 1 69
PROPOSITIONAL LOGIC
Propositional logic
• Logical constants: true, false
• Propositional symbols: P, Q, S, ... (atomic sentences)
• Wrapping parentheses: ( … )
• Sentences are combined by connectives:
...and [conjunction]
...or [disjunction]
AB...implies [implication / conditional]
AB..is equivalent [biconditional]
...not [negation]
• Literal: atomic sentence or negated atomic sentence
Examples of PL sentences
• P means “It is hot.”
• Q means “It is humid.”
• R means “It is raining.”
• (P Q) R
“If it is hot and humid, then it is raining”
• QP
“If it is humid, then it is hot”
• A better way:
Hot = “It is hot”
Humid = “It is humid”
Raining = “It is raining”
Propositional logic (PL)
• A simple language useful for showing key ideas and definitions
• User defines a set of propositional symbols, like P and Q.
• User defines the semantics of each propositional symbol:
• P means “It is hot”
• Q means “It is humid”
• R means “It is raining”
• A sentence (well formed formula) is defined as follows:
• A symbol is a sentence
• If S is a sentence, then S is a sentence
• If S is a sentence, then (S) is a sentence
• If S and T are sentences, then (S T), (S T), (S T), and (S ↔ T)
are sentences
• A sentence results from a finite number of applications of the above
rules
A BNF grammar of sentences in propositional
logic
S := <Sentence> ;
<Sentence> := <AtomicSentence> | <ComplexSentence> ;
<AtomicSentence> := "TRUE" | "FALSE" |
"P" | "Q" | "S" ;
<ComplexSentence> := "(" <Sentence> ")" |
<Sentence> <Connective> <Sentence> |
"NOT" <Sentence> ;
<Connective> := "NOT" | "AND" | "OR" | "IMPLIES" |
"EQUIVALENT" ;
Some terms
A complex sentence:
Models of complex sentences
Inference rules
• Logical inference is used to create new sentences that
logically follow from a given set of predicate calculus
sentences (KB).
• An inference rule is sound if every sentence X produced
by an inference rule operating on a KB logically follows
from the KB. (That is, the inference rule does not create
any contradictions)
• An inference rule is complete if it is able to produce
every expression that logically follows from (is entailed
by) the KB. (Note the analogy to complete search
algorithms.)
Sound rules of inference
• Here are some examples of sound rules of inference
• A rule is sound if its conclusion is true whenever the premise is
true
• Each can be shown to be sound using a truth table
RULE PREMISE CONCLUSION
Modus Ponens A, A B B
And Introduction A, B AB
And Elimination AB A,B
Double Negation A A
Unit Resolution A B, B A
Resolution A B, B C AC
Proving things
• A proof is a sequence of sentences, where each sentence
is either a premise or a sentence derived from earlier
sentences in the proof by one of the rules of inference.
• The last sentence is the theorem (also called goal or
query) that we want to prove.
• Example for the “weather problem” given above.
1 Humid Premise “It is humid”
2 HumidHot Premise “If it is humid, it is hot”
3 Hot Modus Ponens(1,2) “It is hot”
4 (HotHumid)Rain Premise “If it’s hot & humid, it’s raining”
5 HotHumid And Introduction(1,2) “It is hot and humid”
6 Rain Modus Ponens(4,5) “It is raining”
Horn sentences
• A Horn sentence or Horn clause has the form:
P1 P2 P3 ... Pn Q
or alternatively
P1 P2 P3 ... Pn Q
(P Q) = (P Q)
where Ps and Q are non-negated atoms
• To get a proof for Horn sentences, apply Modus Ponens
repeatedly until nothing can be done
• We will use the Horn clause form later
Entailment and derivation
• Entailment: KB |= Q
• Q is entailed by KB (a set of premises or assumptions) if and only if
there is no logically possible world in which Q is false while all the
premises in KB are true.
• Or, stated positively, Q is entailed by KB if and only if the conclusion
is true in every logically possible world in which all the premises in
KB are true.
• Derivation: KB |- Q
• We can derive Q from KB if there is a proof consisting of a sequence
of valid inference steps starting from the premises in KB and
resulting in Q
Propositional logic is a weak language
NFGP- Unit 1 87
Predicate Logic
What is a Logic?
• A language with concrete rules
• No ambiguity in representation (may be other errors!)
• Allows unambiguous communication and processing
• Very unlike natural languages e.g. English
• Many ways to translate between languages
• A statement can be represented in different logics
• And perhaps differently in same logic
• Expressiveness of a logic
• How much can we say in this language?
• Not to be confused with logical reasoning
• Logics are languages, reasoning is a process (may use logic)
18CSC305J_AI_UNIT3 89
11/06/2021
Syntax and Semantics
• Syntax
• Rules for constructing legal sentences in the logic
• Which symbols we can use (English: letters, punctuation)
• How we are allowed to combine symbols
• Semantics
• How we interpret (read) sentences in the logic
• Assigns a meaning to each sentence
• Example: “All lecturers are seven foot tall”
• A valid sentence (syntax)
• And we can understand the meaning (semantics)
• This sentence happens to be false (there is a counterexample)
Propositional Logic
• Syntax
• Propositions, e.g. “it is wet”
• Connectives: and, or, not, implies, iff (equivalent)
• For all X
• if (X is a rose)
• then there exists Y
• (X has Y) and (Y is a thorn)
Example: FOL Sentence
• “On Mondays and Wednesdays I go to John’s house for dinner”