X

i
l
i
n
x

S
i
m
u
l
i
n
k

_________________________
_________________________
_________________________
2013 05 05
Muhammad Arif Mughal

s20111983






Modeling telemetry system for sensor monitoring
based on co-simulation of Xilinx System Generator
and SIMULINK







100083

Master Degree Candidate Haris Anwaar
Supervisor Prof. Yang YuLiang
School of Computer and Communication Engineering
University of Science and Technology Beijing
30 Xueyuan RoadHaidian District
Beijing 100083P.R.CHINA



____________ ______________
____________ ______________





Xilinx Simulink


_________________________








2013 5 5











- I -

I am heartly pleased to express my sincere gratitude to my supervisor Prof.
Yang YuLiang for his guidance, encouragement and constant support throughout
my research work. I commend him giving me freedom for independent research.
My words are inadequate to express my heartful thanks to Prof. Yang YuLiang for
his enthusiastic support in all efforts during my research period. I would like to
express my gratitude to his sincerity, dignity and dedication towards work. He has
been a constant inspiration for me and I feel privileged to have been associated
with him.
I wish to thank my group mate and friend LiuWei for his great cooperation,
his timely and generous help during my research period.
I pay special thanks to all of my classmates, lab mates and country mates for
their help and encouragement for my research work.
Moreover, I would like to express my deep appreciation and love to my
entire family especially my mother and father for their support, prayers, patience
and encouragement during all of my studies living away from them. In the end, I
am deeply thankful to God to guide me and provide me strength to complete my
research work.













Table of Contents

Abstract ................................................................................................................ v
Chapter 1 : I ntroduction ........................................................................................ 1
1.1 Purpose of Project ................................................................................................. 1
1.2 Scope of Research.................................................................................................. 1
1.3 Applications .......................................................................................................... 1
1.4 Overview of Thesis ................................................................................................ 2
Chapter 2 : Literature Survey ................................................................................ 3
2.1 Long Term Wind Energy Prediction ..................................................................... 3
2.1.1 ANN ....................................................................................................................................... 3
2.2 Short Term Wind Power Prediction ...................................................................... 4
2.2.1 Irrelevancy Filter .................................................................................................................... 4
2.2.2 Redundancy Filter .................................................................................................................. 6
2.2.3 RBFNN .................................................................................................................................. 6
2.2.4 EPSO...................................................................................................................................... 7
2.3 SVR ....................................................................................................................... 8
2.4 GRNN ................................................................................................................... 9
Chapter 3 : Implemented Methodologies .............................................................. 11
3.1 Long Term Wind Energy Prediction ................................................................... 11
3.1.1 Data Set ................................................................................................................................ 11
3.1.2 Why Use These Values ........................................................................................................ 11
3.1.3 Grassi Model Implementation .............................................................................................. 12
3.1.4 Improved Models ................................................................................................................. 13
ii


3.2 Short Term Wind Power Prediction .................................................................... 14
3.2.1 Two Stage Feature Selection ............................................................................................... 14
3.2.2 Forecast Engine.................................................................................................................... 15
3.2.3 Data Set ................................................................................................................................ 17
3.3 SVR ..................................................................................................................... 18
3.4 GRNN ................................................................................................................. 18
Chapter 4 : Results and Discussion ...................................................................... 19
4.1 Long Term Wind Energy Prediction ................................................................... 19
4.1.1 Grassi Model Implementation .............................................................................................. 19
4.1.2 Three ANN Model ............................................................................................................... 20
4.1.3 Three ANN with EPSO Model ............................................................................................ 22
4.1.4 Comparison of Three Models .............................................................................................. 23
4.2 Short Term Wind Power Prediction .................................................................... 24
4.3 GRNN ................................................................................................................. 25
4.4 SVR ..................................................................................................................... 26
Chapter 5 : Conclusion and Future Work ............................................................ 28
5.1 Conclusion .......................................................................................................... 28
5.2 Future Work ....................................................................................................... 28
References .......................................................................................................... 30


iii


List of Figures

Figure 2-1 ANN Architecture [4] .......................................................................................................... 3
Figure 2-2 MI [3] .................................................................................................................................... 5
Figure 2-3 RBFNN [7] ............................................................................................................................ 6
Figure 2-4 PSO (particles converges to best position) [4] ................................................................... 7
Figure 2-5 Mapping of input into high dimension [9] ......................................................................... 8
Figure 2-6 SVR [4] .................................................................................................................................. 8
Figure 2-7 GRNN Architecture [11] ................................................................................................... 10
Figure 3-1 ANN architecture [2] ......................................................................................................... 12
Figure 3-2 Three ANN Model [4] ........................................................................................................ 13
Figure 3-3 ANN architecture used in three ANN model [2] ............................................................. 13
Figure 3-4 Three ANN with EPSO Model [4] .................................................................................... 14
Figure 3-5 MHNN and EPSO hybrid model [4] ................................................................................ 15
Figure 3-6 Forecast engine [4] ............................................................................................................. 16
Figure 3-7 ANN Architecture used in MHNN [4] .............................................................................. 17
Figure 4-1 Testing plot of Grassi model implementation .................................................................. 19
Figure 4-2 Training plot of Grassi model implementation ............................................................... 20
Figure 4-3 Training and testing plot given in the paper [2] .............................................................. 21
Figure 4-4 Testing plot of three ANN implementation ...................................................................... 21
Figure 4-5 Training plot of three ANN implementation ................................................................... 22
Figure 4-6 Training plot of three ANN with EPSO implementation ............................................... 23
Figure 4-7 Testing plot of three ANN with EPSO implementation .................................................. 23
Figure 4-8 Testing plot of short term wind power prediction .......................................................... 25
Figure 4-9 Testing plot of GRNN ........................................................................................................ 26
Figure 4-10 Testing plot of SVR .......................................................................................................... 27











iv


List of Tables

Table 4-1 Results on Grassi model Implementation .......................................................................... 19
Table 4-2 Results of three ANN Model ............................................................................................... 20
Table 4-3 Results of three ANN with EPSO Model ........................................................................... 22
Table 4-4 Comparison of models on long term wind energy prediction paper ............................... 24
Table 4-5 Results of short term wind power prediction .................................................................... 24
Table 4-6 Results on GRNN ................................................................................................................ 25
Table 4-7 Results on SVR .................................................................................................................... 26



v


Abstract

Wind power prediction system is essential component in power distribution system.
Wind is a fluctuating source of energy which has raised the issue of reliability of
power distribution system. To ensure reliability, wind power must be predicted
accurately in advance. The purpose of the project is to exp lore the learning
capability of both artificial neural network (ANN) and evolutionary algorithms (EA)
for wind power prediction and then develop a hybrid system of ANN and EA to
improve the efficiency of wind power prediction. Wind power prediction systems
using ANN, general regression neural network (GRNN), support vector regression
(SVR), combination of different types of ANNs and hybrid model of ANNs along
with enhanced particle swarm optimization (EPSO) are explained. It is observed that
hybrid model of ANNs along with EPSO and SVR are showing good results in terms
of accuracy.

1


Chapter 1 : Introduction

One of the most important challenges faced by mankind now days is the issue of
energy. The issue of energy crisis has led the researchers to think and industrialist to
invest in the alternate energy resources. One of alternative source is wind, and it is
gaining popularity because it has the potential to produce power on commercial scale.
But there are some problems regarding wind, wind is a fluctuating source of energy
which has raised the issue of reliability of power distribution system. To ensure
reliability, wind power must be predicted accurately in advance. Wind power can be
estimated on long and short term basis. Short term wind power prediction period varies
from minutes to a day, whereas long term wind power prediction period varies from
months to years. Here both short and long term wind power prediction are considered.
1.1 Purpose of Project
The purpose of the project was to explore the learning capability of both artificial
neural network (ANN) and evolutionary algorithms (EAs) for wind power prediction.
Also to develop a hybrid system of ANN and EAs to improve the efficiency of wind
power prediction.
1.2 Scope of Research
The work presented in thesis is enough to develop a system that can do wind power
forecasting. Different techniques of ANNs, EAs and hybrid system of ANN with EA
along with general regression neural network (GRNN) and support vector regression
(SVR) are studied and also implemented. Efficiency of the wind power prediction was
improved by changing the parameters of the system.
1.3 Applications
Wind power prediction has greater application in electricity supply system, because
without wind power prediction system grid operators are unable to schedule the
economically efficient generation of electricity and also cannot ensures system
reliability of the electricity supply system. This is because of fluctuating and
intermittent behavior of wind power generation. Therefore wind power forecasting is
an important factor of power systems [1].
2


1.4 Overview of Thesis
In second chapter literature survey is discussed. Third chapter will describe the
implementation of a paper [2], which is based on long term wind energy prediction,
and also improvement in its accuracy, and implementation of another paper [3], which
is based on short term wind power prediction, and also implementation of the SVR
and GRNN techniques on the data set, used in the implementation of short term wind
power prediction paper. Fourth chapter will comprised of results and discussion of the
different models explained in the third chapters. Fifth and the last chapter will be on
conclusions and future work.

3


Chapter 2 : Literature Survey

This chapter will build the understanding of the different techniques and components
that are used in the implementation of the papers [2], [3] along with SVR and GRNN.
First of all, techniques and components of paper [2] will be discussed then techniques
and components of paper [3] will be discussed and after that SVR and GRNN will be
discussed.
2.1 Long Term Wind Energy Prediction
Paper [2], which is based on long term wind energy, uses ANN as wind energy
prediction system. Therefore only ANN will be discussed in detailed.
2.1.1 ANN
An ANN is a computational model that is inspired by the structure of human brain
consisting of neurons act as biological neural networks. A neural network consists of
layers of artificial neurons that are connected to each other through weights as shown
in Figure2-1.


Then the neural network is trained for a problem on the data relevant to the problem,
which will assign values to the weights (connection between two artificial neurons).
Now if the trained neural network is subjected to unknown data of same problem it
will give predicted values against the input test data [4].
Input Layer Hidden Layer Output Layer

Figure 2-1 ANN Architecture [4]



4


On basis of the architecture of neural networks there are many algorithms designed to
train the neural network but mostly widely used and efficient algorithm is back
propagation neural network (BPNN). In BPNN error is calculated at each step of
training at output nodes and then back propagates it to adjust the weights [4].
Inputs used for wind power prediction are mainly, wind speed, humidity, air pressure,
temperature, wind direction, generations hours e.tc. Different inputs will be tested on
experimental basis and those will be selected having greater influence on wind power
prediction [5].
BPNN will be used to implement wind power prediction system. The performance
parameters mainly in BPNN are mainly
- Number of hidden layers along with number of neurons
- Transfer functions used
- Number of input units, and
- Which inputs are to use
2.2 Short Term Wind Power Prediction
Wind power prediction system proposed in the paper [3] is composed of feature
selection and forecast engine shown. Feature selection comprises of irrelevancy filter
and redundancy filter and forecast engine contains modified hybrid neural network
(MHNN) with enhanced particle swarm optimization (EPSO). First irrelevancy filter
then redundancy filter will be discussed. But MHNN is combination of ANNs and
radial basis function neural network (RBFNN), so only RBFNN will be discussed. In
the last EPSO will be discussed.
2.2.1 Irrelevancy Filter
The basic idea about this filter is that, it calculates mutual information (MI) value
between the target variable and all the input features one by one, MI basically shows
the common information between the two variables as shown in the Figure 2-2, after
calculating MI values it is decided that which variables should be used in next step on
basis of MI values, variables having MI values greater than specific thresh hold are
selected and others are discarded [6]. The performance of the filter will mainly
depend on the threshold that is used. MI values were calculated by first linearly
normalizing all the candidate features and the output target in the range [0-1].Then
median for each variable is calculated. Values of variable greater than its median were
5


rounded to 1 and values less than its median were rounded to 0.Then MI values were
calculated by using the formula given in equation (2.1)

m
m m 2
m
m
m 2
m
m
m 2
m
P(X=0,Y =0)
MI(X,Y )=P(X=0,Y =0)log
P(X=0)XP(Y =0)
P(X=1,Y =0)
+P(X=1,Y =0)log
P(X=1)XP(Y =0)
P(X=0,Y =1)
+P(X=0,Y =1)log
P(X=0)XP(Y =1)

(
(

(
(

(
(

m
m 2
m
P(X=1,Y =1)
+P(X=1,Y =1)log
P(X=1)XP(Y =1)
(
(

(2.1)
Where,
X=Target
Y
m
=Input candidate features
m m
m
m
m
m
m
m
m
m
m m
m m
m m
m
m m
m
U =2*Y +X
U0
P(X=0,Y =0)=
L
U2
P(X=0,Y =1)=
L
U1
P(X=1,Y =0)=
L
U3
P(X=1,Y =1)=
L
(U0 +U2 )
P(X=0)=
L
(U1 +U3 )
P(X=1)=
L
(U0 +U1 )
P(Y =0)=
L
(U2 +U3 )
P(Y =1)=
L



Figure 2-2 MI [3]







MI(X,Y)
6


2.2.2 Redundancy Filter
The basic idea behind this filter is that higher value of mutual information between
two selected candidates means more common information among them, so they are
redundant candidates. Therefore redundancy of each selected feature with the other
candidate inputs are calculated according to formula given in equation (2.2).

| | | | ( )
x m(t) S1(t) - x k(t)
RC xk(t) =max MI xk(t),xm(t)
e
(2.2)
Then redundancy criterion (RC) values were observed among the input features that
were greater than the thresh hold and discarded one of the input feature on basis of MI
values, candidate having smaller value of MI was discarded [3]. The performance of
the filter will mainly depend on the threshold that is used.
2.2.3 RBFNN
RBFNN have three layers an input layer, hidden layer with nonlinear radial basis
function (RBF) as activation function and a linear output layer as shown in Figure 2-
3. The output of the RBFNN is represented by equation (2.3).

( )
N
i i
i=1
(x)= a x-c

(2.3)
Where, N is the number of neurons in the hidden layer, c
i
is the center of each neuron
in the hidden layer. The norm is taken as Euclidian distance and the basis function is
taken as Gaussian [7].


Figure 2-3 RBFNN [7]
7


2.2.4 EPSO
In particle swarm optimization (PSO) population of candidates (individuals) or swarm
of particles exists, and then these particles are moved in the search space. Each
particle movement in the search space is influenced by its best known position known
as personal best or local best and also by the best known position in the overall search
space known as global best, which are updated as better positions are explored. This
moves the swarm towards the best solutions as shown in the Figure 2-4, and then best
of all candidates or particles is selected as optimal solution [4].

The objective is to minimize the error for which the tness function will be simply the
error function. Each particle in the PSO represents a candidate solution to the
optimization problem; it might be the weights of an ANN or parameters of some other
models. PSO gives the best particle which minimize the error function [4].
EPSO is same as PSO but the difference is that EPSO instead of using global best and
personal best also uses personal-not best component. The equations used are given
below

( ) ( )
( )
best not-best
i,iter+1 i,iter 1b 1 i,iter i,iter 1nb 2 i,iter i,iter
best
2 3 iter i,iter
V =X +c r P -X +c r X -P
+c r G -X
(2.4)

i,iter+1 i,iter i,iter+1
X =X +V (2.5)

best best
i,iter i,iter i,iter i,iter-1
P =X , if OF(X ) < OF(P ) (2.6)

best best best
i,iter i,iter-1 i,iter i,iter-1
P =P , if OF(X ) OF(P ) > (2.7)

not-best best
i,iter i,iter i,iter i,iter-1
P =X , if OF(X ) OF(P ) > (2.8)

not-best not-best best
i,iter i,iter-1 i,iter i,iter-1
P =P , if OF(X ) < OF(P ) (2.9)

Figure 2-4 PSO (particles converges to best position) [4]
8


Equation (2.4) is the velocity formula which moves the particle on basis of local best,
global best and personal-not best components. Equation (2.5) is the formula for
updating the particle position in the space. Equation (2.6) and (2.7) are formulas for
updating the personal best of the particle. Equation (2.8) and (2.9) are formulas for
updating the personal-not best of the particle.
The performance parameters in EPSO case will be factors that derive the
mathematical formula for changing the position of the particle in the swarm.
2.3 SVR
Support vector regressions are used due to many reasons; some of these are usage of
kernels, absence of local minima, sparseness of the solution and capacity control
obtained by acting on the margin etc. SVR follows these points to create a prediction
model [8].First of all SVR transform the input vectors into other dimension by using
kernels as shown in the Figure 2-5.

Figure 2-5 Mapping of input into high dimension [9]
The input vectors in the high dimension mostly act as linear due to sparseness created
by the kernel. Then linear hyper plane is fitted on the data in the other dimension.
Hyper plane is fitted in such a way to get maximum margins on its both sides which
helps in correct prediction as shown in the Figure 2-6. Then with help of the hyper
plane we predict the target value for the input test values.



i

Error
Loss
Support Vectors
i



Figure 2-6 SVR [4]

9


2.4 GRNN
GRNN is one pass learning algorithm and is basically probability based prediction
neural network. Its work can be explained by a simple example given below.
There is a fund in which three peoples are contributing. First one contributed $2,
second one $3 and the third one contributed $100. Then three numbers are calculated,
sum of dollar amount times the person number (1*$2+2*$3+3*$100=$306), sum of
the $ amount ($2+$3+$100=$105), then divide the first number by second (306 / 105
~ 3).This shows the top contributor person in the fund i.e. person number 3. This is
exactly what GRNN does; the pattern that is very close to training data will get heavy
contribution while the other will get very smaller in calculating the output predicted
value [10], [11]. The architecture of GRNN is shown in the Figure 2-7.
- Input layer: The input neurons feed their values into each node of hidden
layer. There is one neuron for each predictor variable in the input layer.
- Hidden layer: In this layer for each value in the training data set one neuron
exists. In each neuron one of the training input vectors along with the target
value is captured or saved.
- Summation layer: There are two neurons in the summation layer. One neuron
is the denominator summation unit and the other is the numerator summation
unit. The denominator summation unit just adds up the weight values coming
from each of the hidden neurons. The numerator summation unit for each
hidden neuron adds up the weight values multiplied by the actual target value
that is saved in the neurons of the hidden layer.
- Decision layer: The decision layer divides the numerator summation unit by
the denominator summation unit and the result is presented as predicted target
value.
The actual working of the GRNN can be explained such as; first the input test vector
is given to input layer which transfer it to each neuron in the hidden layer. In each
neuron of the hidden layer, the distance between the input test vector and the vector
that is already saved in that corresponding neuron is calculated. The rbf activation
function is then applied on the resultant, which gives maximum value if the resultant
is small and minimum value otherwise. The output of the hidden neuron is called
weight. Then the weights are provided to summation layer and then at last to the
decision layer for final prediction.
10




Input Hidden Summation Decision
layer layer layer layer

Figure 2-7 GRNN Architecture [11]

X1
X2
X3
11


Chapter 3 : Implemented Methodologies

This chapter will briefly describe the implementation details of the papers [2], [3]
along with SVR and GRNN. First of all paper [2] will be discussed then paper [3] will
be discussed and after that SVR and GRNN will be discussed.
3.1 Long Term Wind Energy Prediction
Here implementation of paper, Wind energy prediction using a two-hidden layer
neural network [2], is discussed in which two hidden layer neural network is used to
make wind energy prediction using back propagation learning algorithm. The network
was trained on two years of data and then tested on one year data.
3.1.1 Data Set
The data set used, is given in the paper [2] in tabular form. The input parameters used
as input for neural network are
- Wind speed
- Relative humidity
- Temperature
- Generation hours
- Maintenance hours
And the output is wind energy. Total 3 years of data was available, each input
parameter and output have total 12 values for year, because their values are taken as
average over the month. The data have been collected from 43 wind turbines over
period from January 2005 to December 2007.
3.1.2 Why Use These Values
Here the mentioned parameters in the data set section are briefly discussed, which
mainly affect the generation of wind power. It is known that the electrical power
generated by a wind turbine is given by equation (3.1)

3
P=0.5Aw (3.1)
Where,
=Air density
w =Wind speed
12


P = Wind Power
A =Area swept by the turbine
From equation (3.1) it is clear that wind speed has a major inuence on the power
output, because power depends on the cubic value of wind speed w. And also, the
wind power depends on the air density , which is in turn affected by the relative
humidity and the temperature. By using generation and maintenance hours as inputs,
the performance of the system is improved.
First, authors Grassi model implementation is discussed then the improved model
implementations are discussed. In improved models, three ANN and three ANN with
EPSO models are discussed.
3.1.3 Grassi Model Implementation
In the paper [2] a two hidden layer neural network is used to predict the wind power
generation of three wind farms. Three years of proper experimental data is used to
train and test the neural network with back propagation learning algorithm. First two
years data was used for training and last year data was used for testing.
a. Structure of ANN
The neural network used is shown in the Figure 3-1. It have 5 input neurons and 1
output neuron with two hidden layers, where hyperbolic tangent is used as activation
function in the rst hidden layer and the logarithmic sigmoid as second hidden layer
activation function and linear activation function at the output layer. Three neurons
are used in the hidden layers.



Figure 3-1 ANN architecture [2]


Humidity
Temperature
Generation hours
Maintenance hours
Wind energy
Wind speed
13


3.1.4 Improved Models
The results of data set of paper [2] were poor on the Grassi implementation so it was
improved by using a different model explained below.
a. Using Three ANNs
From model of paper [3], a part of the model was used to improve the accuracy,
shown in Figure 3-2.


Explanation of how the model works is in coming sections. The ANN used in this
case has the structure shown in Figure 3-3. Five inputs, seven first hidden layer
neurons, five second layer hidden layer neurons are used. Hyperbolic tangent is used
as activation function in the rst hidden layer and the logarithmic sigmoid as second
hidden layer activation function and linear activation function at the output layer.



Figure 3-3 ANN architecture used in three ANN model [2]

Humidity
Temperature
Generation hours
Maintenance hours
Wind speed
Wind Energy

Figure 3-2 Three ANN Model [4]

14


b. Using Three ANNs with EPSO
The three ANN model was further modified and the results were further improved.
The modified model is given in Figure 3-4. The model shown in the Figure 3-4 is
same as used in paper [3] except that the feature selection component and RBFNN are
not used. The ANN used here is same as used in three ANN model shown in Figure 3-
3, i.e. having five inputs, seven first hidden layer, five second hidden layer neurons
with hyperbolic tangent as first hidden layer, logarithmic sigmoid as second hidden
and linear as output layer activation functions.

3.2 Short Term Wind Power Prediction
Here implementation of paper [3] is discussed briefly. Wind prediction system
proposed in the paper is composed of feature selection and forecast engine shown in
Figure 3-5. Feature selection comprises of irrelevancy filter and redundancy filter and
forecast engine contains MHNN with EPSO.
3.2.1 Two Stage Feature Selection
Wind power can be seen as a nonlinear mapping function of its past values and the
meteorological variables, and these variables are available in wind farm. Set of
candidate features used as input for prediction system is given in equation (3.2)

( ) ( ) ( ) ( )
( ) ( ) ( )
( ) ( ) ( )
( ) ( ) ( ) ( )
( ) ( ) ( ) ( )
wp
ws
wd
H
T
S t ={ WP t-1 , WP t-2 ,... WP t-N ,
WS t ,WS t-1 ,...,WS t-N ,
WD t ,WD t-1 ,...,WS t-N ,
H t ,H t-1 , H t-2 ,... H t-N ,
T t ,T t-1 , T t-2 ,... T t-N }
(3.2)

Figure 3-4 Three ANN with EPSO Model [4]

15



Where,
WP=Wind Power
WD=Wind Direction
WS=Wind Speed
H=Humidity
T=Temperature
N
wp
=N
ws
=N
wd
=N
T
=N
H
=Number of past values to be considered
The input candidate feature set is first pass through irrelevancy filter which filter out
those features that are irrelevant to the output target or not help much in predicting the
output wind power. Then the output of the irrelevancy filter is given to the
redundancy filter which finds and discards those features that are redundant. Working
of both filters has been discussed in literature survey chapter in detail.
3.2.2 Forecast Engine
After applying the two stage feature selection to the candidate feature set, the filtered
set is supplied to the forecast engine to train itself; forecast engine is shown in Figure
3-6. Forecast engine mainly comprises of MHNN and EPSO.
First the data set is given to RBFNN, which gives initial forecast for target variables
after training. Then the initial forecast from RBFNN plus the filtered candidate input
feature set is given to artificial neural network 1 (NN1). NN1 after training, further
tuned its weights through EPSO, then the forecast for target plus the final weights of
NN1 are passed to artificial neural network 2 (NN2). NN2 also train itself on the
filtered input candidate feature set plus the forecast of target provided by NN1,
provided with initial weights that of passed by NN1. NN2 weights are then further

Figure 3-5 MHNN and EPSO hybrid model [4]

16


tuned through EPSO. Then the forecast for target plus the final weights of NN2 are
passed to artificial neural network 3 (NN3). NN3 also train itself on the filtered input
candidate feature set plus the forecast of target provided by NN2, provided with initial
weights that of passed by NN2, the NN3 also further tuned its weights with help of
EPSO. In the testing phase we will need only the weights of NN1, NN2 and NN3. The
selected input feature set will be given first to RBFNN, giving target prediction
values. Then NN1 will be given the selected features along with predicted target
values of RBFNN, giving predicted target values. Similarly NN2 and NN3 will give
target predicted values. Target predicted values of the NN3 will be considered final.

a. ANNs in MHNN
From Figure 3-6 it is clear that each ANN transfers two kinds of results to next ANN.
The rst set of results transferred i.e. weights and bias values are actually the
knowledge of the ANN which it learnt, and the next ANN begin its learning process
from the point that the previous ANN terminated. Here all ANNs of the MHNN have
the same number of output, input and hidden neuron so that next ANN increases the
obtained knowledge of its previous NN. The structure of ANN used is given in Figure
3-7. Activation function used for hidden layer was tangent sigmoid.


Figure 3-6 Forecast engine [4]
17



Three different ANNs have been selected in MHNN because by suitable selection of
training algorithms of the ANNs the MHNN can learn more than a single ANN.
Levenberg-Marquardt (LM) is selected as first ANN because it is a fast learning
algorithm and at the beginning of the training phase it learns quickly about the
problem and its training error quickly decreases. The Broyden-Fletcher-Goldfarb-
Shanno (BFGS) is selected as second ANN because it performs a better search of the
solution space with condition that it starts from a suitable initial point i.e. provided
with good initial weights. The Bayesian Regularization (BR) is selected the third
ANN because it generalizes the problem well [12].
b. EPSO
EPSO is used because when if any ANN is trapped in a local minimum then neither
that ANN nor the next one may be able to escape from the local minimum. The EPSO
is used after each ANN of the MHNN to avoid the explained situation i.e. trapping of
the ANN in local minima.
3.2.3 Data Set
The data set is taken from the website of Morrisville State College [13]. Data set
comprises of data of year 2008,2009,2010,2011 and two months of year 2007. The
data set have average daily values of wind speed, relative humidity, temperature and
wind power. So modified input candidate input feature set is used i.e. without wind
direction, instead of the candidate input feature set that is represented by equation
(3.2). Data from 2007 to 2010 was used for training purpose and 2011 data was used
for testing purpose.

Figure 3-7 ANN Architecture used in MHNN [4]


18


3.3 SVR
SVR was applied on data set taken from Morrisville State College [13]. The data set
was first applied to the irrelevancy filter then to redundancy filter as explained in the
feature selection of paper [3], after that the filtered set was given to the SVR for
training. 25% of the data set was used for testing and 75% for training.
3.4 GRNN
GRNN was applied on data set taken from Morrisville State College [13]. The data set
was first applied to the irrelevancy filter then to redundancy filter as explained in the
feature selection of paper [3], after that the filtered set was given to the GRNN for
training. 25% of the data set was used for testing and 75% for training.



19


Chapter 4 : Results and Discussion

4.1 Long Term Wind Energy Prediction
4.1.1 Grassi Model Implementation
For training and testing the data set was divided into two parts, first two years data
was used for training purpose and the last year was used for testing purpose. For
training, the data was first normalized between 0 and 1, total 500 epochs were run to
train the neural network and the performance function used was MSE (Mean Square
Error).After training the ANN, it was tested on the last year data and the results
obtained are shown in Table 4-1.
Table 4-1 Results on Grassi model Implementation
Training Testing
MAE 0.1351 0.1020

The testing and training plots are shown graphically in Figure 4-1 and Figure 4-2
respectively.

Figure 4-1 Testing plot of Grassi model implementation
20



Figure 4-2 Training plot of Grassi model implementation
In the training plot shown in Figure 4-2 the predicted plot is not completely matching
the actual plot but the predicted shows the general trend. This is because of the
generalization behaviour of the neural network. In the testing plot shown in Figure 4-1
predicted values are close for months 2,3,5,8,10,11 but other values are at some
distance from the original one.The plot given in the paper is shown in Figure 4-3,
comparison of our plot and the plot given in the paper shows that our implementation
results are not good. Reason is that author have not discussed the implementation
completely, we have implemented the paper on some assumptions.
4.1.2 Three ANN Model
Similarly for three ANN model first two years data was used for training purpose and
the last year was used for testing purpose. The results obtained are given in Table 4-2
Table 4-2 Results of three ANN Model
Training Testing
MAE 0.0114 0. 0138

The testing and training plots are shown graphically in Figure 4-4 and Figure 4-5
respectively.
21



Figure 4-3 Training and testing plot given in the paper [2]

Figure 4-4 Testing plot of three ANN implementation
In the the testing plot shown in Figure 4-4 predicted values are close for all months
except the 4 one. In the training plot shown in Figure 4-5 the predicted plot is not the
exact match of the actual plot but the predicted shows the general trend. If the
predicted plot in Figure 4-5 exactly matches the original one then on testing data the
ANN wiil show great error.
22



Figure 4-5 Training plot of three ANN implementation
4.1.3 Three ANN with EPSO Model
Similarly for three ANN model with EPSO first two years data was used for training
purpose and the last year was used for testing purpose. The results obtained are given
in Table 4-3
Table 4-3 Results of three ANN with EPSO Model
Training Testing
MAE 0.0127 0. 0118

The training and testing plots are shown graphically in Figure 4-6 and Figure 4-7
respectively .In the the training plot shown in Figure 4-6 the predicted plot is not the
exact match of the actual plot but the predicted shows the general trend. If the
predicted plot in Figure 4-6 exactly matches the original one then on testing data the
NN wiil show great error because of over fitting problem. In the testing plot shown in
Figure 4-7 predicted values are close for all months except the 4 one. This means the
error value in case of testing is mainly due to the value of the 4 month. The
comparison of these plots with the ones given in the paper have close resemblence,
this is explained in the coming section.
23



Figure 4-6 Training plot of three ANN with EPSO implementation

Figure 4-7 Testing plot of three ANN with EPSO implementation
4.1.4 Comparison of Three Models
Comparison of the results of three models mentioned previously is given in Table 4-4.
From the Table 4-4 it is clear that the results of three ANN model and three ANN
with EPSO model have better results in case of testing than the other two.
24


Table 4-4 Comparison of models on long term wind energy prediction paper
MAE
Grassi Model
Implementation
Three ANN
model
Implementation
Three ANN
with EPSO
model
Implementation
Authors
error
(mentioned
in paper)
Testing 0.1020 0.0138

0. 0118 0.0156
Training 0.1351 0.0114

0.0127 0.0109

From Table 4-4 it can be seen that among the mentioned models the training error is
minimum for the authors model that is mentioned in its paper. It can be seen that the
training error for three ANN model is smaller than that of three ANN with EPSO
model but the testing error of the three ANN with EPSO model is smaller than the
three ANN model error, it is because of the generalization. The three ANN with
EPSO model is showing greater generalization then the three ANN model because it
has not memorized the training data, instead it has learned only the trend from the
data set.
4.2 Short Term Wind Power Prediction
Ninety nine input candidate feature set was given to two stage feature selection, in
which 25 features were of wind speed, 25 features were of relative humidity, 25
features were of temperature and 24 features were of wind power. Five features were
passed from two stage feature selection to the forecast engine. The forecast engine
trained itself on the training data and then the forecast engine was tested for which
result is shown in Table 4-5
Table 4-5 Results of short term wind power prediction
MSE Accuracy %
Testing 0.0072 91.75

The testing plot is shown in the Figure 4-8. From the plot it can be seen that the actual
and the predicted plots are close on the whole region. But if the plot is seen in the
25


between 45-110 (x-axis) region, it can be seen that the actual and predicted values in
that region are not much close to each other i.e. there is error involve for those values.
4.3 GRNN
The data set used is taken from Morrisville State College [13]. The data set was first
applied to the irrelevancy filter in the form of feature set given in equation (3.2), the
irrelevancy filter discarded those features that were irrelevant to the target variable i.e.
wind power. After that the relevant features were passed to redundancy filter which
discarded the redundant features among relevant features. Then the filtered set was
given to the GRNN for training. 25% of the data set was used for testing and 75% for
training. Results obtained are given in Table 4-6
Table 4-6 Results on GRNN
MSE Accuracy %
Testing 0.0426 77.81


Figure 4-8 Testing plot of short term wind power prediction
The testing plot is shown in the Figure 4-9. From the plot it can be seen that the actual
and the predicted plots are comparable in the region from 150 to 328, but in the
remaining region the two plots are at distant from each other. Therefore we can say
that the values contributing to the error are lying in the region 1-150.
26


4.4 SVR
The data set used is taken from Morrisville State College [13]. The data set was first
applied to the irrelevancy filter in the form of feature set given in equation (3.2), the
irrelevancy filter discarded those features that were irrelevant to the target variable i.e.
wind power. After that the relevant features were passed to redundancy filter which
discarded the redundant features among relevant features. Then the filtered set was
given to the SVR for training. 25% of the data set was used for training and 75% for
training. Results obtained are given in Table 4-7
Table 4-7 Results on SVR
MSE Accuracy %
Testing 0.0038 97.62

The testing plot is shown in the Figure 4-10. From the plot it can be seen that the
actual and the predicted plots are almost same on the whole region. But if we see the
plot carefully the values contributing to error are lying in the region 45-75. Hence
SVR is giving much better prediction accuracy, which is essential in wind power
prediction.

Figure 4-9 Testing plot of GRNN
27



Figure 4-10 Testing plot of SVR




28


Chapter 5 : Conclusion and Future Work

5.1 Conclusion
Different models were applied to predict wind power accurately; it includes Grassi
model of ANN, GRNN, SVR and hybrid model of ANN with EPSO along with two
stage feature selection module which comprised of irrelevancy filter and redundancy
filter. Models only involving ANN i.e. Grassi model of ANN and GRNN are not
providing good results of prediction. The main reason behind this was trapping of the
ANN in the local minima i.e. the weight solution provided by the learning algorithm
was sub optimal, which is one of the weaknesses that ANN have. Compare to Grassi
ANN and GRNN models hybrid model of ANN with EPSO along with feature
selection module performed well, because in this model the local minima trapping
problem was almost solved by the EPSO. EPSO has a property that if sub optimal
solution is acquired then in the next step random step is provided to the particles in
the swarm which will help the particles in the swarm to get out of the local minima. In
comparison to all models SVR performed very well, because SVR is free of many
problems that ANN has e.g. trapping in local minima because it is based on structural
risk minimization. From all these it can conclude that SVR model is providing good
results for our problem.
5.2 Future Work
The data set used [13] in short term wind power prediction is taken from the internet.
Since there are also wind farms on short scales in Pakistan that are operating in some
regions, the work presented in this thesis can be checked on those stations and their
performance can be evaluated in real time situations. Since the operating stations of
wind farm in Pakistan are not of commercial scale, the models that are mentioned
here require some historical data for training so that they can predict wind power for
future. Therefore, by working on wind power prediction system from earlier along
with the other works on wind farms will help a lot in the time when the stations will
opened for commercial purposes. Additionally, the work presented here mostly
comprises of predicting wind power on short term basis because most of the time
short term wind power predictions are needed, but there are also situations where we
29


need long term wind power predictions. The models that are presented here can also
work for long term wind power prediction we only have to change the data set on
which the model is trained. Therefore in future these models can be tested for long
term wind power predictions and changes can be made in order to achieve high
accuracy.




30


References


[1] M. C. Mabel and E. Fernandez, Analysis of wind power generation and
prediction using ANN: A case study, Renewable Energy, Vol. 33, pp. 986992,
2008.
[2] G. Grassi and P. Vecchio, Wind energy prediction using a two-hidden layer
neural network, Communications in Nonlinear Science and Numerical
Simulation, Vol. 15, pp. 22622266, 2010.
[3] N. Amjady, F. Keynia, H. Zareipour, Wind Power Prediction by a New
Forecast Engine Composed of Modied Hybrid Neural Network and Enhanced
Particle Swarm Optimization, IEEE Transactions on Sustainable Energy, Vol.
2, no. 3, pp. 265-276, July 2011.
[4] Andries P. Engelbrecht, Computational Intelligence an Introduction, 2
nd
Edition,
John Wiley & Sons Inc., 2007.
[5] R. Jursa and K. Rohrig, Short-term wind power forecasting using evolutionary
algorithms for the automated specification of artificial intelligence models,
International Journal of Forecasting, Vol. 24, pp. 694 709, 2008.
[6] N. Amjady and F. Keynia, Day-Ahead Price Forecasting of Electricity Markets
by Mutual Information Technique and Cascaded Neuro-Evolutionary
Algorithm, IEEE Transactions on Power Systems, Vol. 24, pp. 306-318,
February 2009.
[7] Wikipedia, Radial Basis Function Neural Network [Online]. Available
http://en.wikipedia.org/wiki/Radial_basis_function_network
[8] Banyan Research Express, Support vector regression [Online]. Available:
http://research.ncku.edu.tw/re/articles/e/20080620/3.html
[9] Kyushu University, Efficient Learning Algorithms for Support Vector
Machines [Online]. Available:
http://www-kairo.csce.kyushu-u.ac.jp/~norikazu/research.en.html
[10] DTREG, Probabilistic and General Regression Neural Networks [Online].
Available: http://www.dtreg.com/pnn.htm
[11] Artificial Intelligence Webring, General Regression Neural Networks
[Online]. Available:
31


http://www.lwebzem.com/computer_science_algorithms/2011/10/09/general-
regression-neural-networks/
[12] N. Amjady, A. Daraeepour, F. Keynia, Day-ahead electricity price forecasting
by modied relief algorithm and hybrid neural network, IET General on
Transmission Distribution, Vol. 4, pp. 432444, 2010.
[13] Morrisville State College, Data Set
[Online].Available:http://www.morrisville.edu/alternativeenergy/windturbine.as
px
[14] Broomhead, D. S.; Lowe, David (1988). Radial basis functions, multi-variable
functional interpolation and adaptive networks (Technical report). RSRE. 4148.
[15] Broomhead, D. S.; Lowe, David (1988). "Multivariable functional interpolation
and adaptive networks". Complex Systems 2: 321355.
[16] Schwenker, Friedhelm; Kestler, Hans A.; Palm, Gnther (2001). "Three
learning phases for radial-basis-function networks". Neural Networks 14: 439
458.
[17] Park, J.; I. W. Sandberg (Summer 1991). "Universal Approximation Using
Radial-Basis-Function Networks". Neural Computation 3 (2): 246257.
doi:10.1162/neco.1991.3.2.246. Retrieved 26 March 2013.
[18] J. Moody and C. J. Darken, "Fast learning in networks of locally tuned
processing units," Neural Computation, 1, 281-294 (1989). Also see Radial
basis function networks according to Moody and Darken
[19] T. Poggio and F. Girosi, "Networks for approximation and learning," Proc.
IEEE 78(9), 1484-1487 (1990).
[20] Roger D. Jones, Y. C. Lee, C. W. Barnes, G. W. Flake, K. Lee, P. S. Lewis,
and S. Qian, ?Function approximation and time series prediction with neural
networks,? Proceedings of the International Joint Conference on Neural
Networks, June 17-21, p. I-649 (1990).
[21] Martin D. Buhmann (2003). Radial Basis Functions: Theory and
Implementations. Cambridge University. ISBN 0-521-63338-9.
[22] Yee, Paul V. and Haykin, Simon (2001). Regularized Radial Basis Function
Networks: Theory and Applications. John Wiley. ISBN 0-471-35349-3.
[23] John R. Davies, Stephen V. Coggeshall, Roger D. Jones, and Daniel Schutzer,
"Intelligent Security Systems," in Freedman, Roy S., Flein, Robert A., and
32


Lederman, Jess, Editors (1995). Artificial Intelligence in the Capital Markets.
Chicago: Irwin. ISBN 1-55738-811-3.
[24] Simon Haykin (1999). Neural Networks: A Comprehensive Foundation (2nd
edition ed.). Upper Saddle River, NJ: Prentice Hall. ISBN 0-13-908385-5.
[25] S. Chen, C. F. N. Cowan, and P. M. Grant, "Orthogonal Least Squares
Learning Algorithm for Radial Basis Function Networks", IEEE Transactions
on Neural Networks, Vol 2, No 2 (Mar) 1991.