Short-Term Load Forecasting Methods in Comparison:
Kohonen Learning, Backpropagation Learning, Multiple Regression Analysis
and Kalman Filters
T. Baumann, H. Strasser
SIEMENS AG Austria
Vienna, Austria
Keywords: Short-term load forecasting, Artificial neural networks, Regression Techniques
1 ABSTRACT
In this paper we present a comparison between four dif-
ferent short-term load forecasting methods. Two of them
are based on artificial neural networks and two on classi-
cal statistical methods. The first method is based on the
well known Backpropagation learning method. The sec-
ond method is based on a combination of the self-orga-
nizing Kohonen algorithm and the supervised delta rule.
The third method, the multiple regression analysis, uses
the technique of weighted least squares estimation,
whereas the last method is based on Kalman filter theory.
2 INTRODUCTION
If in the last few years, the short time load forecasting has
become more and more important, new load forecasting
techniques have been created or adapted
The artificial neural network (ANN) approach is only a
new step in this evolution, which began in the forties by
discovering the strong correlation between the load and
meteorological data.
Nowadays, approximately 25 papers have been published
on the load forecasting using ANN. But only a few of
them did compare their method to classical forecasting
methods [1], [2], [3] and [4]. In this paper, the most pop-
ular ANN approach, the Multilayer Perceptron with
backpropagation learning, two statistical methods [5], the
Multiple Regression Analysis and Kalman filter and a
new ANN method are compared.
The basic idea of this new approach has been presented
in (6). It consists of a combination of a self-organizing
ANN, the Kohonen Feature Map, and a single layered lin-
ear network with delta rule learning. This approach is ex-
plained in detail, while the other three methods will only
briefly be discussed and references to existing work are
provided.
A neural network is a collection of simple interconnected
processing units or neurons. The processing units operate
independently from each other, thus, realizing an archi-
tecture that is parallel and distributed, resulting in a high
speed information processing and a potential fault toler-
ance. The connection weights that determine the way in
which neurons interact can be modified during the net-
445
H. Landrichter
Wiener Elektrizitatswerke
Vienna, Austria
works operation providing a high degree of adaptability.
This adaptation process is the so-called learning.
In contrast to classical software, which are programmed
by a succession of instructions, the ANNs work with a
training process. In other words, an algorithm is not pro-
grammed anymore, but the ANN is trained by a succes-
sion of examples (Input I Output pairs). Therefore, this
approach is useful for tasks where the basic laws are not
known or only poorly known, which is e.g. the case for
load forecasting.
In the last decade, interest in neural networks has revived
as advances in computer technology have made it easier
to simulate large networks in software. At present there
exist numerous of different architectures of ANN.
Among them, the best adapted one must be found for a
particular task.
3 FOUR FORECASTING METHODS
3.1 ANN wjth BacJwro:oagatjon Learning
The Multilayer Perceptron (MLP) with backpropagation
learning is the most popular neural algorithm. The meth-
od is based on a supervised learning algorithm. During
the learning phase, the neural network learns a nonlincar
mapping from a n-dimensional input vector space to
m-dimensional output vector space ~n-> ~m, by mini-
mizing an energy function, calculated from the mean
square error between the actual ANN outputs and the de-
sired outputs. A general structure of an MLP is shown in
Fig. 1. For more detail see also [7] and [8].
The short-term load forecasting method with MLP pre-
sented in this paper is similar to the methods explained in
[8] and (9]. Two main changes are proposed: introduction
of \Yeather data and the use of seven MLP's, one MLP for
each day of the week. Therefore, the resulting seven neu-
ral networks need following inputs: the load and weather
data of the present day and forecasted weather data of the
day for which the load is to be forecasted. For the latter
day, the network then outputs the loads ..
ASSOCIATION \
~ Y: Output
uCD
>
Fig. 1 Structure of an MLP with one hidden layer.
Therefore, the input vector contains 144 components:
48 half-hourly loads(day)
24 temperature( day)
24 light(day)
24 temperature( day+ 1)
24 light(day+l)
and the output vector contains 48 values:
48 half-hourly loads(day+ 1)
All the inputs and outputs were normalized to [0.1, 0.9].
The seven MLPs were composed of an input layer with
144 neurons, two hidden layers with 60 and 48 neurons
and an output layer with 48 neurons. The following pa-
rameters were used for learning: learning rate = 0.6, mo-
mentum term 0.3. The learning was performed until the
system error fell below 0.003.
There are many other possibilities to design an MLP ar-
chitecture. Alternatively, non fully connected MLP's, and
a different organization of inputs and outputs could be
tested with the same data set.
3.2 Kohonen/delta method
This method consists of two phases: the basic load fore-
casting and the weather dependent load correction (Fig.
6). In Phase 1, the load is calculated without any weather
data influence using the self-organized Kohonen network
(section 3.2.1).
In Phase 2, a one layered ANN is trained to produce a
weather dependent correction of the loads obtained in
Phase 1 (section 3.2.5).
3.2.1 The Kohonen network
In 1982, Teuvo Kohonen proposed a new neural network
architecture that implements a self organizing algorithm
(10], (11].
As rioted above, an ANN can be characterized as a collec-
tion of processing elements (PE), also called neurons,
which can all perform the same task in parallel. The in-
formation to be processed is distributed via single-direc-
446
tional connections (weights or synaptic weights) to all
PE's.
Xf Inputj
Wf Weightj
W: Weight vector
y X: Input vector
Fig. 2 The processing element or artificial neuron
Fig. 2 shows a schema of a PE where each in11ut X).~s con-
nected to the PE via a synaptic weight W;. X = lX 1, X_i,
.. Xn) is called input vector and on the otfier hand the W
=CW1. W2 •.. Wn) is called weight vector.
For most ANN the basic operation using Xand Wto cal-
culate the output Y is the scalar product For the Kohonen
network, the basic operation is the Euclidian distance:
N
= DEuct = L <Xr ~)
2
Y
j=1
where N is the number of components of the input vector.
The basic architecture of the Kohonen network is com-
posed of an input layer with N inputs followed by a layer
with M * M neurons, arranged in a two-dimensional lat-
tice, also called map(Fig. 3). Each input is connected
through a weight Wij to all neurons of the second layer.
Fig. 3 Basic architecture of the Kohonen network.
(Connections are only shown partially)
The training of an ANN is done by a continuous modifi-
cation of the weights Wij• following an internal law which
differs from network to network. This needs a repeated
presentation of input vectors until the ANN is organized.
A single learning step can be summarized by the two fol-
lowing phases:
1. Find the neuron c whose weight vector is nearest
to the input vector. (i.e. Find the best-matching
neuron c)
An input vector X is presented to the Kohonen network
and the Euclidian distan~ between this input vector and
all the weight vectors W; (i = 1 .. M2) is calculated.
Therefore an Euclidian distance is calculated for each
neuron. The neuron with the smallest distance is the win-
ner (also called active ~uron). This neuron gets the index
c. The weight v~tor We is consequently the nearest to
the input vector X. This phase can be summarized by: •
DEuc/W0 X) = min(DEuctWi. X)) for i = 1... /.I
the nearest neuron to the (incomplete) input vector is
computed. The weights between this selected neuron and
all the ignored input components are then the values of
the missing components of the input vector. In Fig. 5, the
Kohonen network is trained with four consecutive load
values, L(t-2) until L(t+ 1), and the recall is done with
three load values and with the load L(t+l) is missing.

2. Update the unit c and its neighbors.

Training Auto-associative memory
Once the winner has been located, unsupervised learning L(t+l)
proceeds by updating the weights of neurons within a
neighborhood Ne defined around winner neuron c (Fig. L(t-2) L(t-2
4).
L(t-1) L(t-1

~ L(t) L(t)

~
Fig. 4 Neighbourhood around the wirmer neuron
All other weight vectors are not modified. In the begin-
ning, the radius of the neighborhood Ne is big (approxi-
mately half of the network size) but then decreases with
the learning step until it reaches zero. If the radius is zero,
only the neuron c is updated. Updating is done as follows:
w,{t+ 1>= Wi<t) + a(t> x (X(O - w.<o>
w.<t+ 1> = W.<t>
where cx(t) is the gain term (0 < cx(t) < 1) which decreases
as a function of the training iteration step, as well as the
neighbourhood.
The key point is that the algorithm performs a reduction
of dimensionality. In our case the dimension of the input
vector space is reduced to two, arranging vectors on a sel-
f-organizing feature map.
Kohonen maps partition the input vector space into M 2
sub-spaces, also called "receptive fields", which are de-
termined by the set of inputs selecting the same neuron.
This mapping is done, conserving the topology of the in-
put vectors. In other words, if two partitions of the input
vector space are "neighbours", then the two correspond-
ing units of the Kohonen map are topological neighbours,
too.
Fig. 5 Principle of auto-associative memory
3 2.3 Application of auto-associative memory to load
forecasting (phase })
An application of auto-associative memory with the Ko-
honen network to single day term load forecasting is de-
scribed. This type of ANN is well suited for the task of
load forecasting due to its characteristic of feature extrac-
tion of input vectors.
During this feature extraction, the Kohonen network cre-
ates (= self-organizes) a fixed number of reference vec-
tors corresponding each to a day of the year (reference
day). Each neuron stores a reference day in its weights.
Therefore, the number of reference days is fixed by the
number of neurons in a Kohonen map. We used 100 neu-
rons networks for all our simulations.
Due to the different load shapes of the seven days in a
week, we used seven Kohonen maps, as well. The type of
the day could also be introduced as input to the neural
network, but our experience gave better results by choos-
ing one Kohonen map for each day type. The inputs, the
desired outputs after learning and the seven Kohonen
maps are shown as Phase 1 in Fig. 6.

3.2.2 Application of the Kohonen network to

auto-associative memory

Once the training is done, it is possible to apply a vector

with missing components and the Kohonen network can
recover the complete vector. This is called auto-associa-
tive memory. For example: when the Kohonen network
has been trained with binary images and only a part of an
image is presented to the Kohonen network, it will recon-
struct the whole binary image.
This method consists of ignoring the missing components
of the input vector and calculating the Euclidian distance Fig. 6 The data flow of the forecasting method.
with the known components only. As in the normal case,

447
 ~ Phase 1

lt~=~l -
[~
ISa_Su l
:1~~~~~1

K1ij+1)]~:i.
2ij+1) ~!;'

r .
: 8;
-g~

~8ij+1) ~~
Phase 2

Mo_Tu= Monday to Tuesday, ect.

The seven ANN's are trained with one to three years of
load data, depending on the availability. During training
the input vector to the Kohonen maps is composed of 96
inputs (48 half-hourly loads of day j + 48 half-hourly
loads of day j+l). When forecasting, only the first 48 in-
puts are presented and the Kohonen network finds the
missing 48 loads of the next day by auto-associative re-
call.
3.2.4 Trend correction
If the seven Kohonen networks are trained with several
years, the produced reference days, which are stored in
the neurons of the Kohonen maps will statistically con-
tain load values which are too small , due to the growing
load over years. The larger the data period to train the
ANN's, the more important this statistical effect will be.
We propose a simple trend correction method to over-
come this problem.
The difference between the load input to the ANN and the
48 weights of the best-matching neuron c is added to the
intermediate output Yi (calculated with auto-association)
of the ANN. This can be summarized by:
~. hase 1 = Y1.+o · (X.J- 48 -Wc.j-48 )
1
forj' =49 ... 96
3.2.5 Introduction of weather data (Phase 2)
Simulations show that Phase I with trend correction per-
forms already well. But the forecast of days with impor-
tant temperature changes between two days gives very
bad results. Now a second neural network is introduced
to correct the forecasted load as a function of weather
data. The basic idea is to relate the errors produced with
Phase I to weather data. For this purpose an ANN is
trained to predict the errors of Phase I as a function of
weather data and then add it to the output of Phase 1 (Fig.
6, Phase 2). Therefore, the input to the ANN is the weath-
er data and the desired outputs are the load errors pro-
duced by Phase 1. The weather data is composed of the
temperature of today and the predicted temperature of the
day to be forecasted. Intuitively, the ANN should learn a
zero correction, if the temperature of the following days
does not change in comparison with the present day.
For that purpose we used a one-layered linear ANN,
trained with the delta rule [7] :
N
net ~
o; = L.., xj' wij recall
j= 1
desired net
= TJ · ( O; - o; ) · xj learning rule
!J,. W;j

where o is a correction factor between 0 and 1. With o= where o?et is the network output of unit i, xj the j1h input,
0.75 we obtained best results. In Fig. 6 the calculation of tJ,. W;; the changes of weights between two learning steps,

trend corrected loads is included in Phase I. o;desrred th e deslf

This method is also useful, if rapid load profile changes
occur (e.g. summer vacation). After one day, the method
adapts the output to the new circumstances.
. ed output an d rt t he 1earnmg
. rate. Note
that the weights wij are different from the weights in the
Kohonen maps.
At present, only one ANN is used for the seven days. In
the future it will be tested, if using more ANN's will im-
prove the performance. Simulations showed that the

448
training data should be composed of the one or two last
months of the actual and last year.
3.3 Multiple regression analysis
By means of multiple regression analysis the statistical
relations between total load and weather conditions as
well as day-type influences are calculated. In other
words, the method attempts to reconstruct analytically
the causal functional relationship between the load and
the influencing factors.
In this paper, the regression coefficients are computed by
an exponentially weighted-least-squares estimation
based on a linear regression model for a given point of
time (e.g. time interval 8.00-8.30 a.m.). The analysis pe-
riod was fixed to five weeks.
The result of this process is a set of polynomial-coeffi-
cients, so called regression coefficients lli· By means of
dummy variables, day-type specific load influences are
taken into account Different day types are commonly
treated in the data analysis in order to take advantage of
the increased amount of statistical data.
The result of the analysis is a decomposition of the time
dependent total load Y(t) into:
* time-sensitive load component
weather insensitive base load B(t)
* weather-sensitive load components
temperature dependent load aT * T(t)
light intensity dependent load aL * L(t)
* day type specific load component D(t)
Y(t) =B(t) + aT * T(t) + aL * L(t) + D(t) + model_error(t)
In the model we used the derived indoor temperature in-
stead of the measured outdoor temperature. The indoor
temperature is calculated by solving the heat flow equa-
tion. The light intensity is transformed via Weber-Fech-
ners-law [5].
3.4 Kalman filter
Multiple regression analysis and the Kalman filter are
parts of the linear filter theory: one considers a linear
combination of certain model functions with unknown
coefficients. In case of multiple regression analysis these
coefficients are time independent, whereas the Kalman
filter uses time evaluated ones (dynamic state equation).
Of course, both methods can be performed with arbitrary
model functions, but the simulations in this paper were
based on linear functions.
The decomposition of the time dependent load into com-
ponents as well as the transformations on temperature
and light intensity are as described in the multiple regres-
sion analysis section.
449
In comparison with the multiple regression analysis, the
Kalman filter adapts itself more rapidly due to the usage
of the last predicted state vector, the last forecast error as
well as the current and the predicted weather data to cal-
culate the next forecast.
A critical point with Kalman filters is the optimal weight-
ing of the different influencing parameters. For example,
a too strong influence of the light intensity may increase
the forecasting errors. Furthermore, the optimal weight-
ing normally change over time, depending on the sea-
sons.
The multiple regression analysis as well as the Kalman
filter approach are explained in detail in [5].
4 RESULTS
The comparison of results are demonstrated using the
half hourly loads and the corresponding weather data
from the electric utility in Vienna (WEW) between the
22. October 91 and 20. October 92. The maximal load is
approximately 1500 MW.
The following error definitions were used:
Joacf!esired - load.
e~el = / /
Relative error
' loa<f!esired
I
ref 1 """Ab ref
MAPE µAbs= "'j( £.,. s ( ei )
I
where MAPE means Mean Absolute Percent Error.
We present results based on a forecasting period of two
weeks, from Monday 21. September to Sunday 4. Octo-
ber 1992. For all methods, the forecasting is done for a
single day at midnight
This test period has been chosen in order to evaluate the
methods in two different ways: first, the forecast perfor-
mance after a long sunny period, and second, the adapting
ability after a change from summer-time (ST) to winter-
time (WT) on 26(27 September.
This change introduces a change in the load shape. There-
fore, within this testing period, the first six days test the
forecasting ability and the last eight days the ability to
adapt to a new situation. This is done for all methods.
For the two ANN methods, the training was performed
with recorded data from the 22. October 91 until one day
before the testing period. The forecasting of the two
weeks is then performed with the previously trained
ANN's. For a longer testing period a retraining of the
neural networks would have been necessary.
For the two classical methods, an adaptation of the meth-
od coefficients was performed with five weeks until one
day before the forecasting day. Therefore the two ANN
methods are slightly handicapped compared to the classi-
cal methods, because due to the high calculation time,
only one learning is done for the whole test period.
The table below show the MAPE calculated over a single Due to the trend correction, the Kohonen/Delta rule
day for all methods over the whole testing period of two method performed by far best during the whole WT peri-
weeks. od. It would be interesting to include a trend correction
algorithm to the other methods, as well.
Date MLP Koh MR.egr Kalman ST/WT
Fig. 7 - Fig. 10 show the effective and forecasted half--
21.9.92 1.94 2.04 1.4 2.50 ~1
hourly loads in MW of the four methods on 30. Septem-
22.9.92 1.32 1.58 1.0 0.79 ST ber 92. They show how fast the four different methods
23.9.92 1.86 0.80 1.0 1.16 ST could adapt to the new situation for one example.
24.9.92 1.60 0.73 0.8 0.85 ST
25.9.92 1.58 0.77 1.4 1.11 ST
26.9.92 1.42 1.78 1.8 1.58 ST
27.9.92 4.94 3.98 4.9 3.96 wr
28.9.92 1.81 2.02 1.5 2.30 wr ~
29.9.92 2.62 0.85 2.0 2.04 wr ~
30.9.92 3.43 1.07 1.5 1.07 wr
1.10.92 2.71 0.96 2.2 1.49 wr
2.10.92 2.81 1.17 2.1 1.06 wr
3.10.92 3.47 1.78 3.9 2.72 wr
4.10.92 6.8 2.21 5.0 5.53 wr
MAPES'! 1.62 1.28 1.23 1.33 s·1 Fig. 7 Effective and forecasted loads of the MLP method.
MAPEWf 3.57 1.75 2.89 2.52 wr
MAPE total 2.74 1.55 2.18 2.01 all

Table 1: Daily MAPEs of the four methods

4.1 ST test period

This period is characterized by the fact that the load

shapes are similar to the preceding 5 weeks. In this peri-
od, the two statistical methods and the Kohonen/Delta
method performed well. The best performance was
achieved by the multiple regression which could take ad- Fig. 8 Effective and forecasted loads of the Kohonen/Del-
vantage of the non changing 5 weeks analysis period. The ta rule method.
MLP method performed the worst.

4.2 wr test period

~
From Saturday 26. Sept to Sunday 27. Sept. the time II ~
changes from ST to WT. Therefore the four methods have
to cope with a new situation. The sunday was forecasted
badly by all the methods. But then, the four methods re-
acted differently to ST/WT change. While the Kohonen/-
Delta method reacted fast, the multiple regression and es-
pecially the MLP method reacted slowly or not. The MLP Fig. 9 Effective and forecasted loads of the multiple re-
method should know this situation, because WT load gression method.
shapes of passed year are included in the data set.
The Kalman filter adapted rapidly to the new situation.
After 3 days it performed better than the multiple regres-
sion method.
The two classical methods mode led the last day very bad- II
ly. This day was the first cloudy day after a clear period.
Therefore, the methods could not correctly model the in-
fluence of the light intensities. The Kohonen/Delta rule
method predicted this day best due to a longer learning
period. The MLP method does not react to the light
change.
Fig. 10 Effective and forecasted loads of the Kalman filter
method.

450
The change from ST to WT induces a strong change in
the load shape around sunset For the 30. September the
Kalman filter and the Kohonen/Delta method have the
same MAPE. While the Kalman filter performs better af-
ter midnight when the loads are small, the Kohonen/Delta
method provides a better forecast around sunset. The
maximum MAPE for this day is 4.34% for the Kalman
filter method and 3.17% for the Kohonen/Delta method.
4.3 Two weeks test oeriod
Over the whole test period, the Kohonen/Delta-rule
method was found out to be the most appropriate load
forecasting technique, especially when load shapes were
changing. This method has also been tested with data of
a different European electric utility and two testing peri-
ods of one month. One period in summer and one in win-
ter. The two-months MAPE was 1.6% [12].
5 CONCLUSIONS
In this paper four different load forecasting techniques
have been compared. Two ANN and two statistical ap-
proaches. The Kohonen/Delta rule approach gave the
best results during a two weeks testing period. This meth-
od adapted very fast to a fairly new situation (changing
from summer-time to winter-time). Another ANN meth-
od, the Multilayer Perceptron with backpropagation
learning gave the worst results. The MLP method, as pro-
posed here, is probably not well adapted, because the data
set is very small (one year). It is known, that the MLP
with backpropagation learning is very sensitive to learn-
ing and network parameters Oeaming rate, hidden layers,
...). Improvements should be possible by changing the pa-
rameters and the structure of the ANN. Furthermore, it
would be interesting to add a trend correction algorithm
to the MLP method. The two statistical methods per-
formed well during the first six days, but both reacted
slowly after the change from summer-time to winter-time
(especially the multiple regression method).
6 REFERENCES
l) Dillon, T.S., Morsztyn, K., Phua, K., "Short Term
Load Forecasting Using Adaptive Pattern Recogni-
tion and Self-Organizing Techniques'', Proc. of
Fifth Power Systems Computation Conference,
Cambridge, Paper 2.4/3, pp. 1-16, 1975.
2) Brace, M.C., Schmidt, J., Haldin, M., "Comparison
of the Forecasting Accuracy of Neural Networks
with other Established Techniques", IEEE, Proc.
First International Forum on Applications of Neural
Networks to Power Systems, Seatle, Washington,
pp. 31-35, July 23-26, 1991.
3) Chen, S-T, Yu, D.C., Moghaddamjo, A.R., "Wea-
ther Sensitive Short-Term Load Forecasting Using
451
Nonfully Connected Artificial Neural Network",
IEEE Transactions on Power Systems, Vol. 7, No. 3,
pp. 1098-1105, August 1992.
4) Srinivansan, D., Liew, A.C., Chen John S.P., "Short
Term Forecasting Using Neural Network Ap-
proach", IEEE, Proc. First International Forum on
Applications of Neural Networks to Power Sy-
stems, Seatle, Washington, pp. 22-25, July 23-26,
1991.
5) H. Strasser, N. Friemelt, G. Schellstede, "Short term
load forecast using multiple regression analysis or
adaptive regression analysis", Proceedings of
PSCC, Graz, 1990.
6) Germond, A.J., Macabrey, N., Baumann, T., "Ap-
plication of Artificial Neural Networks to Load Fo-
recasting", INNS:Workshop on Neural Network
Computing for the Electric Power Industry, Stan-
ford University, Stanford, California, August 17-19,
1992.
7) Rumelhart, D.E., Hinton, G.E., Williams, R.J., "Le-
aming Internal Representations by Error Propagati-
on", In Parallel Distributed Processing, chap 8, pp.
318-362, MIT Press, Cambridge, 1986.
8) Lee, K.Y., Cha, Y.T.,Park, J.H., "Short-Term Load
Forecasting using an Artificial Neural Network",
IEEE Transactions on Power Systems, Vol 7, No. 1,
pp. 124-130, February, 1992.
9) Lee, K.Y., Cha, Y.T., Ku, C.C., "A Study on Neural
Networks for Short-Term Load Forecasting", IEEE,
Proc. First International Forum on Applications of
Neural Networks to Power Systems, Seatle, Was-
hington, pp. 26-30, July 23-26, 1991.
10) Kohonen, T., "Self-Organization and Associative
Memory", 3rd edition, Springer Verlag, Berlin,
1989.
11) Kohonen, T., "Self-Organized Formation of Topo-
logical Correct Feature Map", Biological Cyberne-
tics, vol. 43, pp. 59-69, 1982.
12) Baumann, T., Germond, AJ., "Application of the
Kohonen Network to Short"Term Load Fore-
casting", submitted to the second International For-
um on Applications of Neural Networks to Power
Systems, April 19-22, Yokohama, Japan, 1993.