Professional Documents
Culture Documents
Advances in DIP: (Fuzzy, Artificial Neural Networks, Expert System and Image Segmentation)
Advances in DIP: (Fuzzy, Artificial Neural Networks, Expert System and Image Segmentation)
Fuzzy
ORIGIN: Aristotle some 2000 years ago formulated Conventional Logic as the Law of Excluded Middle (ie., Crisp Set) , which has dominated Western Logic ever since.. But the era was over.. Lotfi Zadeh, Professor, University of California at Berkley,had first conceived the idea of Fuzzy Logic during 1960s as a way of processing data by allowing partial set membership rather than crisp set membership.
Fuzzy sets:
Fuzzy Membership
Membership Functions
A membership function (MF) is a curve that defines how each point in the input space is mapped to a membership value (or degree of membership) between 0 and 1. The input space is sometimes referred to as the universe of discourse, a fancy name for a simple concept. It associates a weighting with each of the inputs that are processed, define functional overlap between inputs, and ultimately determines an output response. The rules use the input membership values as weighting factors to determine their influence on the fuzzy output sets of the final output conclusion.
fuzzy
Alternative Notation
Fuzzy-Set Operations
expressed using membership functions
1
A B
0
Fuzzy OR (union) Fuzzy AND (intersection)
Fuzzy Classification
Method In traditional classification, information is represented in a one-pixel -one - class method and its classification result is definite, i.e. a pixel belongs to a class. In fuzzy classification, a pixel is considered having different membership grades obtained from fuzzy classification indicate the area proportion of every cover classes in a mixed pixel. As the traditional method, the fuzzy classification can also be divided into fuzzy supervised and fuzzy unsupervised classification.
where n is the total number of sample pixels, fc is membership of a pixel to class c, xi is pixel value of sample pixels. Fuzzy covariance matrix can be expressed as
After the fuzzy parameters are determined, every pixel is classified according to its spectral value. To perform a fuzzy classification, the membership function must be defined for each class, In this work, the membership functions are defined based on maximum likelihood classification algorithm with fuzzy and fuzzy covariance matrix replacing the conventional mean and covariance matrix. The following is the definition of membership function for cover class c: N is the pixel
vector dimension, m is number of predefined classes.
The Neuron:
The basic neuron consists of; Synapses: are connections between neurons they are not physical connections, but miniscule gaps that allow electric signals to jump across from neuron to neuron. Soma: these electrical signals are then passed across to the soma which performs some operation and sends out its own electrical signal to the axon. Axon: the axon then distributes this signal to dendrites. Dendrites: carry the signals out to the various synapses,and the cycle repeats
Artificial Neuron: Each neuron has a certain number of inputs. Each of which have a weight assigned to them. The weights simply are an indication of how important the coming signal for that input is. The net value of the neuron is then calculated. The net is simply the weighted sum, the sum of all the inputs multiplied by their specific weight. Each neuron has its own unique threshold value, and if the net is greater than the threshold, the neuron fires(or output a 1), otherwise it stays quiet(outputs a 0). The output is then fed into all the neurons it is connected to.
Architecture of Neural Network: The network consists of several layers of neurons An input layer Hidden layer Output layer Hidden layer-because the user cannot see the inputs or outputs for those layers. Ups and downs of neural network: They are excellent as pattern classification/recognizers. Neural Network can handle exceptions and abnormal input data. The power of neural network lies in their ability to process information in a parallel fashion but machines today are serial they only execute on instruction at a time.
An artificial neuron is a model whose components have direct analogs to the components of actual neuron. The input signal are represented by x0, x1, ..xn. Each of these inputs is modified by a weight (synaptic weight) whose function is analogous to that of the synaptic junction in a biological neuron. These weights can be either positive or negative. The processing element consists of two parts The first part simply sums the weighted input resulting in a quantity I. The second part is effectively a non-linear filter, usually called the activation function through which the combined signal flows. Activation function be; It can be threshold function that passes information, when the output of the first part of the artificial neuron exceeds the threshold T.
(I) = 1, I>T 0, I<=T
Perceptron: Perceptron is the most basic term used in ANN. It can be single layered or multi-layered structure. Single perceptron: The structure of a single perceptron is very simple. There are two inputs, a bias and an output In the perceptron, the most common form of learning is by adjusting the weights by the difference between the desired output and the actual output. i = xi = desired output actual output.
Linear associator neural network: The most elementary neural network is a linear associator The fundamental neuron sums the weighted inputs and then subjects this sum to a nonlinear activation function to keep the output with in reasonable range. The architecture of a linear associator is a set of input neurons that are connected to a set of output neurons. (i.e., a two =layer neural network) When the activation functions of the neurons are linear, this network is a linear associator It is not very accurate, if two many items are stored in the associators.
Learning: This talk about weights and thresholds. Which leads to an obvious question, how are all these set? Most popular training methods are; Back-propagation The delta rule Kohonen learning Learning rules can be categorized into two areas: Supervised Un-supervised Supervised learning rules require a teacher. Unsupervised learning do not require teacher because they produce their own output, which is then further evaluated.
Learning and Recall: Learning is the process of adapting the connection weights in an artificial network to produce the desired output vector presented to the input buffer. Recall is the process of accepting an input stimulus and producing an output response in accordance with the network weight structure. Recall is an integral part of the learning process since a desired response to the network must be compared to the actual output to create an error function. In supervised learning, the artificial neural network is trained to give the desired response to a specific input stimulus. In graded learning, the output is graded as good or bad on a numerical scale, and the connection weights are adjusted in accordance with the grade. In unsupervised learning there is no specific response sought, but rather the response is based on the networks ability to organize itself.
Back Propagation:
Back-propagation is a systematic method for training multiple (three or more) layer artificial neural networks. The problem with the perception is that it cannot express non-linear decisions. The perceptron is basically a linear threshold device, which returns a certain value, 1 for example , if the dot product of the input vector and the associated weight vector plus bias surpasses the threshold, and another value, -1 for example, if the threshold is not reached. For example: If the dot product of the input vector and the associated weight plus the bias, is graphed it is obviously linear. More over, all the input vectors that will give a value greater than the threshold are separated into one space, and those that will not, will be separated into another.
Linear plot
Non-linear plot
Multi-layer feed forward network normally consist of three or four layers; with one output and minimum one hidden layer. The raw values of input layer are fed to hidden layer. Once the neurons for the hidden layer are computed, their activations are then fed downstream to the next layer, until all the activations eventually reach the output layer, in which each output layer neuron is associated with a specific classification category. Thus in computing the values of each neuron in the hidden and output layers one must first take the sum of the weighted sums and the bias, and then apply sigmoid function to calculate the neurons activation. Then how does the network learn the problem? The solution is by modifying all the weights.
How to modify the weights; Take partial derivative of the error of the network with respect to each weight, we will learn a little about the direction the error of the network is moving. If the derivative is positive, this tells us that the error is increasing when the weight is increasing, then add a negative value to the weight and vice versa if the derivative is negative. Because the taking of these partial derivatives and then applying them to each of the weights takes place starting from the output layer to hidden layer weights, then the hidden layer to input layer weights. This algorithm has been called the Back Propagation Algorithm.
Basic steps in back-propagation training for a multi-layer neural network: 1. Randomize the weights to small random values (both positive & negative) to ensure that the network is not saturated by large values of weights.(if all weights start at equal values, and desired performance requires unequal weights, the network would not train at all) Select a training pair from the training set. Apply the input vector to network input. Calculate the network output. Calculate the error, the difference between the network output and the desired output. Adjust the weights of the network in a way that minimizes this error. Repeat steps 2 6 for each pair of input output vectors in the training set until the error fro the entire system is acceptably low.
2. 3. 4. 5. 6. 7.
Neural Networks
A NN is a machine learning approach inspired by the way in which the brain performs a particular learning task: Knowledge about the learning task is given in the form of examples.
Inter neuron connection strengths (weights) are used to store the acquired information (the training examples). During the learning process the weights are modified in order to model the particular learning task correctly on the training examples.
Learning
Supervised Learning Recognizing hand-written digits, pattern recognition, regression. Labeled examples (input , desired output) Neural Network models: perceptron, feed-forward, radial basis function, support vector machine. Unsupervised Learning Find similar groups of documents in the web, content addressable memory, clustering. Unlabeled examples (different realizations of the input alone) Neural Network models: self organizing maps, Hopfield networks.
Network architectures
Three different classes of network architectures
single-layer feed-forward multi-layer feed-forward recurrent neurons are organized in acyclic layers
The architecture of a neural network is linked with the learning algorithm used to train
Input layer
Output layer
Hidden Layer
Recurrent network
Recurrent Network with hidden neuron(s): unit delay operator z-1 implies dynamic system
z-1
z-1
The Neuron
The neuron is the basic information processing unit of a NN. It consists of: 1 A set of synapses or connecting links, each link characterized by a weight: W1, W2, , Wm 2 An adder function (linear combiner) which computes the weighted sum of the inputs:
u = w jx j
3
Activation function (squashing function) for limiting the amplitude of the output of the neuron.
j =1
y = (u + b)
The Neuron
Bias
x1
b w1
Local Field Activation function
Input signal
x2
w2
()
Output
M
xm
M
wm
Synaptic weights
Summing function
Bias of a Neuron
Bias b has the effect of applying an affine transformation to u
v=u+b
v is the induced field of the neuron
u = w jx j
j=1
x0 = +1 x1
Input signal
w0 w1 w2
v=
wx
j=0
j j
w0 = b
x2
weights
Local Field
Activation function
()
Output
M
xm
wm Synaptic
Summing function
Segmentation
is the process of dividing an image into regions of related content. is a process of partitioning an image into some non-overlapping meaningful homogeneous regions process of subdividing or partitioning of an image into homogeneous (with respect to some criterion of homogeneity) and disjoint (non-overlapping, non uniform) regions of different statistical behaviour, called image segments
Segmentation
Grouping together pixels that have similar properties such as color, texture, motion, etc Each pixels can be treated as a data point in the feature space
Segmentation
These image regions or segments may be later associated with information labels, but the segmentation process simply gives each region a generic label (region 1, region 2, etc.). In the context of earth remote sensing, these labels would generally be a ground cover type or land use category.
The shape of an object can be described in terms of: Its boundary requires image edge detection The region it occupies requires image segmentation in homogeneous regions, Image regions generally have homogeneous characteristics (e.g. intensity, texture)
Segmentation Algorithms
Segmentation algorithms are based on one of two basic properties of intensity values - discontinuity and similarity. First category is to partition an image based on abrupt changes in intensity, such as edges in an image. Second category are based on partitioning an image into regions that are similar according to a predefined criteria. Histogram Thresholding approach falls under this category.
Segmentation Approaches
Thresholding Boundary Detection Region Growing
Thresholding
Suppose that the gray-level histogram corresponds to an image, f(x,y), composed of dark objects in a light background, in such a way that object and background pixels have gray levels grouped into two dominant modes. One obvious way to extract the objects from the background is to select a threshold T that separates these modes. Then any point (x,y) for which f(x,y) > T is called an object point, otherwise, the point is called a background point.
Segmented Image
Segmentation
Based on Similarity measures
Region Growing Splitting Split and Merge
Region Growing
A simple approach to image segmentation is to start from some pixels (seeds) representing distinct image regions and to grow them, until they cover the entire image For region growing we need a rule describing a growth mechanism and a rule checking the homogeneity of the regions after each growth step
Split / Merge
The opposite approach to region growing is region shrinking ( splitting ). It is a top-down approach and it starts with the assumption that the entire image is homogeneous If this is not true , the image is split into four sub images This splitting procedure is repeated recursively until we split the image into homogeneous regions
Split / Merge
If the original image is square N x N, having dimensions that are powers of 2(N = 2n): All regions produced but the splitting algorithm are squares having dimensions M x M , where M is a power of 2 as well (M=2m,M<= n). Since the procedure is recursive, it produces an image representation that can be described by a tree whose nodes have four sons each Such a tree is called a Quadtree.
Split / Merge
Quadtree
R0
R1 R0 R3 R1
R2
R00
R01
R02
R04
Split / Merge
Splitting techniques disadvantage, they create regions that may be adjacent and homogeneous, but not merged. Split and Merge method It is an iterative algorithm that includes both splitting and merging at each iteration:
Expert Systems
Knowledge Based Information System (KBIS) Expert System (ES): A KBIS that uses its knowledge about a specific area to act as an expert consultant to the end user
Expert Systems
Expert Systems and DSS:
ES
are built into a DSS to improve the quality of decision making; helps add structure to unstructured problems. Support a specific problem domain. Uses a knowledge database to support decision making.
[Figure 11.7]
Advantages of ES
Serve in the absence of a real expert. Provide consistent advice, unaffected by stress and time constraints. Knowledge can be transferred and reproduced. May be less expensive than an actual expert, especially if the knowledge is used over and over.