Introduction

Neurofuzzy Network

Neural Networks
Lesson 9 Fuzzy Logic
Prof. Michele Scarpiniti
INFOCOM Dpt. - Sapienza University of Rome
http://ispac.ing.uniroma1.it/scarpiniti/index.htm
michele.scarpiniti@uniroma1.it

Rome, 26 November 2009

M. Scarpiniti

Neural Networks Lesson 9 - Fuzzy Logic

1 / 37

Introduction
Neurofuzzy Network

Introduction
Basic Concepts
Fuzzy Logic

Neurofuzzy Network
Neurofuzzy Network
Fuzzy Clustering

M. Scarpiniti

Neural Networks Lesson 9 - Fuzzy Logic

2 / 37

Introduction
Neurofuzzy Network

Basic Concepts
Fuzzy Logic

Introduction

Introduction

M. Scarpiniti

Neural Networks Lesson 9 - Fuzzy Logic

3 / 37

Introduction
Neurofuzzy Network

Basic Concepts
Fuzzy Logic

Introduction to Fuzzy Logic

Fuzzy logic is a form of multi-valued logic to deal with reasoning that is

approximate rather than precise.
Fuzzy logic emerged as a consequence of the 1965 proposal of fuzzy
set theory by Lotfi Zadeh. Though fuzzy logic has been applied to
many fields, from control theory to artificial intelligence, it still
remains controversial among most statisticians who prefer Bayesian
logic, and some engineers who prefer traditional two-valued logic.
A logic based on the two truth values True and False is sometimes
inadequate when describing human reasoning. Fuzzy logic uses the
whole interval between 0 (False) and 1 (True) to describe human
reasoning.

M. Scarpiniti

Neural Networks Lesson 9 - Fuzzy Logic

4 / 37

Introduction
Neurofuzzy Network

Basic Concepts
Fuzzy Logic

Fuzzy sets
Fuzzy sets are a further development of the mathematical concept of a set.
A set is any collection of objects which can be treated as a whole. Cantor
described a set by its members, such that an item from a given universe is
either a member or not. A set can be specified by its members, they
characterize a set completely. The list of members A = {0, 1, 2, 3}.
Nobody can list all elements of an infinite set, we must instead state some
property which characterizes the elements in the set, for instance the
predicate x > 10.
Following Zadeh many sets have more than an either-or criterion for
membership. Take for example the set of young people. A one year old baby
will clearly be a member of the set, and a 100 years old person will not be a
member of this set, but what about people at the age of 20, 30, or 40 years?
Another example is a weather report regarding high temperatures, strong
winds, or nice days. Zadeh proposed a grade of membership, such that the
transition from membership to non-membership is gradual rather than
abrupt.
M. Scarpiniti

Neural Networks Lesson 9 - Fuzzy Logic

5 / 37

Introduction
Neurofuzzy Network

Basic Concepts
Fuzzy Logic

Fuzzy sets
The grade of membership for all its members thus describes a fuzzy set. An items
grade of membership is normally a real number between 0 and 1, often denoted .
The higher the number, the higher the membership. Zadeh regards Cantors set as
a special case where elements have full membership = 1. He nevertheless called
Cantors sets nonfuzzy , but today the term crisp set is used.

Note that a fuzzy membership function is different from a statistical probability

distribution. A possible event does not imply that it is probable. However, if it is
probable it must also be possible. You might view a fuzzy membership function as your
personal distribution, in contrast with a statistical distribution based on observations.
M. Scarpiniti

Neural Networks Lesson 9 - Fuzzy Logic

6 / 37

Introduction
Neurofuzzy Network

Basic Concepts
Fuzzy Logic

Fuzzy sets
Notice that Zadeh does not give a formal basis for how to determine the grade
of membership. The membership for a 50 year old in the set young depends on
ones own view. The grade of membership is a precise, but subjective measure that
depends on the context.

Elements of a fuzzy set are taken from a universe. The universe contains all
elements that can come into consideration and it depends on the context. For
example the number between 0 and 100, or, for non-numerical quantity, bitter,
sweet, sour, salt, hot,....
M. Scarpiniti

Neural Networks Lesson 9 - Fuzzy Logic

7 / 37

Introduction
Neurofuzzy Network

Basic Concepts
Fuzzy Logic

Membership function
Every element in the universe of discourse is a member of the fuzzy set to
some grade, maybe even zero. The set of elements that have a non-zero
membership is called the support of the fuzzy set. The function that ties a
number to each element x of the universe is called the membership function
(x). There are two alternative ways to represent a membership function:
continuous or discrete.
In the continuous form the membership function is a mathematical
function. A membership function is for example bell-shaped (also
called a -curve), s-shaped (called an s-curve), a reverse s-curve
(called z-curve), gaussian, triangular, or trapezoidal.
In the discrete form the membership function and the universe are
discrete points in a list (vector). Sometimes it can be more
convenient with a sampled (discrete) representation.
A fuzzy set is normalized if its largest membership value equals 1.
M. Scarpiniti

Neural Networks Lesson 9 - Fuzzy Logic

8 / 37

Introduction
Neurofuzzy Network

Basic Concepts
Fuzzy Logic

Membership function: example

Following some well-known membership functions are reported.
Triangular:


 
x a c x
(x) = max min
,
,0
ba c b

Trapezoidal:


 
x a d x
(x) = max min
,
,0
ba d b

M. Scarpiniti

Neural Networks Lesson 9 - Fuzzy Logic

9 / 37

Introduction
Neurofuzzy Network

Basic Concepts
Fuzzy Logic

Membership function
Other well-known membership functions are reported.
Gaussian:
(x a)2
(x) = exp
2b 2


Bell-shaped:
(x) =

1
 xa 2b

1+
c

M. Scarpiniti

Neural Networks Lesson 9 - Fuzzy Logic

10 / 37

Introduction
Neurofuzzy Network

Basic Concepts
Fuzzy Logic

Singletons
A fuzzy set A is a collection of ordered pairs
A = {(x, (x))}
Item x belongs to the universe and (x) is its grade of membership in A. A single
pair (x, (x)) is called a fuzzy singleton. Thus the whole set can be viewed as the
union of its constituent singletons. It is often convenient to think of a set A just
as a vector
a = [(x1 ), (x2 ), . . . , (xn )]

M. Scarpiniti

Neural Networks Lesson 9 - Fuzzy Logic

11 / 37

Introduction
Neurofuzzy Network

Basic Concepts
Fuzzy Logic

Linguistic variables
Just like an algebraic variable takes numbers as values, a linguistic variable takes
words or sentences as values. The set of values that it can take is called its term
set. Each value in the term set is a fuzzy variable defined over a base variable.
The base variable defines the universe of discourse for all the fuzzy variables in the
term set. In short, the hierarchy is as follows: linguistic variable fuzzy variable
base variable.
Example
Let x be a linguistic variable with the label Age. Terms of this linguistic variable are
from the term set
T = {Old, VeryOld, NotSoOld, MoreOrLessYoung , QuiteYoung , Young , VeryYoung }
Each term is a fuzzy variable defined on the base variable, which might be the scale
from 0 to 100 years.

A primary term is a term or a set that must be defined a priori, for example
Young and Old, whereas the sets VeryYoung and NotYoung are modified.
M. Scarpiniti

Neural Networks Lesson 9 - Fuzzy Logic

12 / 37

Introduction
Neurofuzzy Network

Basic Concepts
Fuzzy Logic

Linguistic variables: an example

Given the set X = {1.5, 1.6, 1.75, 1.8, 2}, we can define the following characterization:
1 short = {1.5/1, 1.6/0.8, 1.75/0.5, 1.8/1, 2/0};
2 normal = {1.5/0, 1.6/0.5, 1.75/1, 1.8/1, 2/0};
3 tall = {1.5/0, 1.6/0.2, 1.75/0.5, 1.8/0.8, 2/1};

M. Scarpiniti

Neural Networks Lesson 9 - Fuzzy Logic

13 / 37

Introduction
Neurofuzzy Network

Basic Concepts
Fuzzy Logic

Operations on Fuzzy sets

Like for the traditional sets, some elementary operations can be developed for
fuzzy sets too.
Complement:
A (x) = 1 A (x)

Intersection:
AB (x) = min (A (x), B (x))

Union:
AB (x) = max (A (x), B (x))

M. Scarpiniti

Neural Networks Lesson 9 - Fuzzy Logic

14 / 37

Introduction
Neurofuzzy Network

Basic Concepts
Fuzzy Logic

Fuzzy logic
The fuzzy logic invalidate two of the strongholds of the classical logic:
1

Law (or principle) of the excluded third (or Tertium non datur in
Latin):


= True
AA

Principle of contradiction (or principium contradictionis in Latin):

= False
AA

In this way the reasoning for fuzzy logic is different from the classical one:
Usually natural and artificial languages have some rules, of the type:
if then
For example:
if it is raining then we get wet;
if the pressure is high then the volume is small.
M. Scarpiniti

Neural Networks Lesson 9 - Fuzzy Logic

15 / 37

Introduction
Neurofuzzy Network

Basic Concepts
Fuzzy Logic

Fuzzy reasoning
In the fuzzy reasoning the linguistic variable contained in a rule, are defined
by fuzzy sets:
if x is A then y is B
A and B are fuzzy sets defined on the universe X and Y ;
the values x X and y Y ;
the fuzzy rule defines a relation R on the space X Y ;
a fuzzy relation is a fuzzy set defined on several domains:
R = X Y Z ...

R (x, y , z, . . .)

For example: binary relation

R =AB
M. Scarpiniti

R (x, y ) = min (A (x), B (y ))

Neural Networks Lesson 9 - Fuzzy Logic

16 / 37

Introduction
Neurofuzzy Network

Basic Concepts
Fuzzy Logic

Fuzzy rules
A fuzzy rule is defined as a conditional statement in two forms:
1

Mamdami rules:

IF x1 is A1 and x2 is A2 and . . . and xN is AN THEN y1 is B1 and y2 is

B2 and . . . and yM is BM
2

Takagi-Sugeno rules:

IF x1 is A1 and x2 is A2 and . . . and xN is AN THEN

y = fk (x1 , x2 , . . . , xN )
The part included between the words IF and THEN is called antecedent, the last
part, after the word THEN, is called consequent. The single assignment xi is
Ai or yi is Bi is called atom.
Example
IF temp IS HIGH AND press IS LOW THEN lever IS UP
M. Scarpiniti

Neural Networks Lesson 9 - Fuzzy Logic

17 / 37

Introduction
Neurofuzzy Network

Basic Concepts
Fuzzy Logic

Inference principle: modus ponens and modus tollens

Given two domains X and Y and a fuzzy relation R, and let us pose the fuzzy sets
A defined on X and B defined on Y , then how can I produce a fuzzy reasoning?
Classical logic uses two fundamental inference principles:
Modus Ponens
Premise: if (x is A) then (y is B)
Antecedent: x is A

Consequent: y is B
Modus Tollens
Premise: if (x is A) then (y is B)
Antecedent: y is not B

Consequent: x is not A

M. Scarpiniti

Neural Networks Lesson 9 - Fuzzy Logic

18 / 37

Introduction
Neurofuzzy Network

Basic Concepts
Fuzzy Logic

Fuzzy inference principle: generalized modus ponens

The modus ponens from standard logical propositional calculus cannot be used in
the fuzzy logic environment causing such an inference can take place if, and only
if, the fact or premise is exactly the same as the antecedent of the IF-THEN rule.
In fuzzy logic the generalized modus ponens is used. It allows an inference when
the fact is only partly known or when the fact is only similar but not equal to it.
Generalised Modus Ponens (for fuzzy logic)
Premise: if (x is A) then (y is B)
Antecedent: x is A0

Consequent: y is B 0
B 0 (y ) = max (min (A0 (x), R (x, y )))
X

Example
Premise: if the tomato is red then it is sweet, possibly sweet-sour, and likely to be sour.
Antecedent: The tomato is more or less red (Red = 0.8).

Consequent: Taste = ?
M. Scarpiniti

Neural Networks Lesson 9 - Fuzzy Logic

19 / 37

Introduction
Neurofuzzy Network

Basic Concepts
Fuzzy Logic

Fuzzy inference principle: a first example

As first example we show an inference with one rule and one atom:
C 0 (y ) = max (min (A0 (x), R (x, y )))
X

where R (x, y ) = min (A (x), C (y )). Hence, we have

C 0 (y ) = maxX (min (A0 (x), A (x), C (y ))) =
= min {maxX (min (A0 (x), A (x))) , C (y )} = min (q0 , C (y ))

M. Scarpiniti

Neural Networks Lesson 9 - Fuzzy Logic

20 / 37

Introduction
Neurofuzzy Network

Basic Concepts
Fuzzy Logic

Fuzzy inference principle: a second example

As a second example we show an inference with one rule and two atoms:

C 0 (y ) = maxX (min (A0 (x1 ), B 0 (x2 ), R (x1 , x2 , y ))) =

= maxX (min (A0 (x1 ), B 0 (x2 ), A (x1 ), B (x2 ), C (y ))) =
= min {maxX1 (min (A (x1 ), A0 (x1 ))) , maxX2 (min (B (x2 ), B 0 (x2 ))) , C (y )
= min (qx10 , qx20 , C (y ))

M. Scarpiniti

Neural Networks Lesson 9 - Fuzzy Logic

21 / 37

Introduction
Neurofuzzy Network

Basic Concepts
Fuzzy Logic

Fuzzy inference principle: a third example

As a third example we show an inference with two rules and two atoms each:

M. Scarpiniti

Neural Networks Lesson 9 - Fuzzy Logic

22 / 37

Introduction
Neurofuzzy Network

Basic Concepts
Fuzzy Logic

Defuzzification
Defuzzification is the process of producing a quantifiable result in fuzzy logic. A
fuzzy quantity is converted into a crisp one (a single number). Defuzzifier is the
system implementing the defuzzification. Several methods are implemented for
defuzzification.
1 Centroid point:
R
x(x)dx
xC = Rx
(x)dx
x
2

Mean of the maximum:

xmed =

N
1 X
MAX (xi )
N
i=1

Minimum Max:
xM = min {MAX (xi )}
X

M. Scarpiniti

Neural Networks Lesson 9 - Fuzzy Logic

23 / 37

Introduction
Neurofuzzy Network

Neurofuzzy Network
Fuzzy Clustering

Neurofuzzy Network

Neurofuzzy Network

M. Scarpiniti

Neural Networks Lesson 9 - Fuzzy Logic

24 / 37

Introduction
Neurofuzzy Network

Neurofuzzy Network
Fuzzy Clustering

Analogies and differences between circuits and neurofuzzy

networks
The fuzzy model can be seen as a digital circuit, designed for signal processing.
Hence there exist an analogy between the fuzzy model and a digital circuit, shown
in the following table:
Circuits
components
topology
synthesis

Fuzzy Model
rules
reasoning
clustering and rules inserting

The neurofuzzy network can be thought as a generalization of the classical digital

circuits, but there exist a substantial difference, show in the following table:
Circuits
indirect information

Fuzzy Model
direct information

The linguistic information can be used directly into the neurofuzzy network: this
possibility is not done for the digital circuits, where these properties must be kept
into account with particular data transformations.
M. Scarpiniti

Neural Networks Lesson 9 - Fuzzy Logic

25 / 37

Introduction
Neurofuzzy Network

Neurofuzzy Network
Fuzzy Clustering

Neurofuzzy networks
The term neurofuzzy refers to combinations of artificial neural networks and fuzzy
logic. Neurofuzzy hybridization results in a hybrid intelligent system that synergies
these two techniques by combining the human-like reasoning style of fuzzy systems with the learning and connectionist structure of neural networks. Neurofuzzy
hybridization is widely termed as Fuzzy Neural Network (FNN) or Neuro-Fuzzy
System (NFS) in the literature. Neurofuzzy system incorporates the human-like
reasoning style of fuzzy systems through the use of fuzzy sets and a linguistic model
consisting of a set of IF-THEN fuzzy rules. The main strength of neurofuzzy systems is that they are universal approximators with the ability to solicit interpretable
IF-THEN rules.
The strength of neurofuzzy systems involves two contradictory requirements in
fuzzy modeling: interpretability versus accuracy. In practice, one of the two properties prevails.
We can show two type of neurofuzzy systems: the first type following the
Mamdamis rules and the second one following the Takagi-Sugenos rules.

M. Scarpiniti

Neural Networks Lesson 9 - Fuzzy Logic

26 / 37

Introduction
Neurofuzzy Network

Neurofuzzy Network
Fuzzy Clustering

Neurofuzzy networks
A neurofuzzy network of the first type is depicted in the following figure:
the output of this network is an expansion of the input following some basis
given by the rules.

M. Scarpiniti

Neural Networks Lesson 9 - Fuzzy Logic

27 / 37

Introduction
Neurofuzzy Network

Neurofuzzy Network
Fuzzy Clustering

Neurofuzzy networks
A neurofuzzy network of
the second type is depicted
in the following figure: it is
an example of a first order
function f (x). This kind
of neurofuzzy network realize an input-output relation as a linear piecewise approximation with
smoothing at the border.
In fact each rule is locally
defined on a small region
of the input data.

M. Scarpiniti

Neural Networks Lesson 9 - Fuzzy Logic

28 / 37

Introduction
Neurofuzzy Network

Neurofuzzy Network
Fuzzy Clustering

Synthesis of neurofuzzy networks

The main problem in designing neurofuzzy systems is the determination of
the fuzzy rules. The specification of such rules are established by determining high density regions in data space using clustering algorithms. Each
rule corresponds to each cluster.
Synthesis of a neurofuzzy networks is based on a clustering algorithm.
In particular the membership function is evaluated on the basis of the corresponding cluster density: after choosing a shape for the membership function, its parameters are determined by its density. A possible form is the
following:
1
(x) =
,
a > 0, b > 0
k2b
1 + kxv
a
where the distance between input data x and v can be measured in several
ways.
M. Scarpiniti

Neural Networks Lesson 9 - Fuzzy Logic

29 / 37

Introduction
Neurofuzzy Network

Neurofuzzy Network
Fuzzy Clustering

Synthesis of neurofuzzy networks

A generic algorithm for the determination of he neurofuzzy network, as
describe in the previous slide, is proposed below:
Synthesis algorithm
1

choosing a membership function shape;

applying a clustering algorithm to the training set;

associating to each cluster a membership function;

learning the membership function parameters proportionally to the

corresponding cluster density;

projecting the clusters to the axis of the input space, determining the
fuzzy values Aik for the rules antecedent;

using the proposed network obtaining the crisp result y after

defuzzification.

M. Scarpiniti

Neural Networks Lesson 9 - Fuzzy Logic

30 / 37

Introduction
Neurofuzzy Network

Neurofuzzy Network
Fuzzy Clustering

Fuzzy Clustering

In fuzzy clustering, data elements can belong to more than one cluster, and
associated with each element is a set of membership levels. These indicate
the strength of the association between that data element and a particular
cluster.
We can modify well-known clustering algorithm in order to obtain
new fuzzy version of this algorithm. Another procedure is to consider
several procedures valid for fuzzy sets and develop new clustering
algorithm.
We will show two clustering algorithms, one for each category:
1
2

M. Scarpiniti

Fuzzy C-means algorithm;

Min-Max algorithm.

Neural Networks Lesson 9 - Fuzzy Logic

31 / 37

Introduction
Neurofuzzy Network

Neurofuzzy Network
Fuzzy Clustering

Fuzzy C-means
Given as examples the N vectors Xk , k = 1, 2, . . . , N, the C-means algorithm is summarized below:
Fuzzy C-means algorithm
1

choosing C vectors Vi (c = 1, 2, . . . , C ) as centroids;

for each input vector Xk choosing a membership function ik (x);

evaluate the new centroids

Vi0

PN

k=1
= P
N

m
ik Xk

k=1

m
ik

i = 1, 2, . . . , C

with m an integer greater than 1.

4

evaluating the sum of the distance of the new centroid from the old ones:
E =

C
X
0

Vi Vi
i=1

if E < with a small positive and arbitrary threshold stop the algorithm, else
repeat from step 2.

M. Scarpiniti

Neural Networks Lesson 9 - Fuzzy Logic

32 / 37

Introduction
Neurofuzzy Network

Neurofuzzy Network
Fuzzy Clustering

Fuzzy Min-Max
The Min-Max clustering algorithm uses a neurofuzzy network called MinMax and is based on the subdivision of the data space into hypercubes
(HC) defined by a couple of points.
1

When a new data point is presented the network evaluates, looking to

its neighborhood, if this point can be included in a present hypercube
with an expansion.

This neighborhood is measured by a membership function.

If there no exist such an hypercube, a new one is created and its

vertexes are defined by the data point Xk .

If two or more hypercubes are partially overlapped, we must contract

these HCs, deciding to which HC a data point belongs.

M. Scarpiniti

Neural Networks Lesson 9 - Fuzzy Logic

33 / 37

Introduction
Neurofuzzy Network

Neurofuzzy Network
Fuzzy Clustering

Fuzzy Min-Max
An example of Min-Max clustering algorithm is reported in the following
figure.

M. Scarpiniti

Neural Networks Lesson 9 - Fuzzy Logic

34 / 37

Introduction
Neurofuzzy Network

Neurofuzzy Network
Fuzzy Clustering

Example: Fuzzy controlling of distance

Let us consider a fuzzy controller for controlling a distance between two cars traveling on a road. The following figure demonstrates how the braking force B depends
upon the distance D and the velocity v . It is assumed that the distance D is
constant, and only the mapping, B = f (v ) , is modeled.

The membership functions are reported below

M. Scarpiniti

Neural Networks Lesson 9 - Fuzzy Logic

35 / 37

Introduction
Neurofuzzy Network

Neurofuzzy Network
Fuzzy Clustering

Example: Fuzzy controlling of distance

The considered rules are the following ones
IF velocity is low, THEN braking force is small.
IF velocity is medium THEN braking force is medium.
IF velocity is high, THEN braking force is large
while the relation B = f (v ) is the following

M. Scarpiniti

Neural Networks Lesson 9 - Fuzzy Logic

36 / 37

Introduction
Neurofuzzy Network

Neurofuzzy Network
Fuzzy Clustering

References

L.A. Zadeh.
Fuzzy sets.
Information and Control, Vol. 8, pp. 338-353, 1965.
V. Kecman.
Learning and Soft Computing. Support Vector Machines, Neural Networks and Fuzzy Logic
Models
The MIT Press, 2001.
O. Maimon and L. Rokach.
Soft Computing for Knowledge Discovery and Data Mining.
Springer, 2008.

M. Scarpiniti

Neural Networks Lesson 9 - Fuzzy Logic

37 / 37