The Use of Artificial Neural Networks in

Nonlinear Microwave Devices and Circuits

Modeling: An Application to
Telecommunication System Design (Invited Article)

´ ´ E. Ngoya, D. Barataud, J. P. Teyssier

Y. Harkouss, J. Rousset, H. Chehade,
IRCOM, Faculte´ des sciences, UMR CNRS no 6615, 123 Av. Albert Thomas, 87060 Limoges Cedex,
´
France; e-mail: harkous@ircom.unilim.fr
20 July 1998; re¨ ised 25 No¨ ember 1998

ABSTRACT: The design of telecommunication systems is a hierarchical process involving

use of a large set of simulation tools and relying on appropriate modeling of system
elements to obtain get-it-right-the-first-time fabrication. This paper investigates in detail
possible application of neural networks to modeling of large-signal hard-nonlinear behavior
of power transistors for circuit design purpose, and modeling of nonlinear circuits such as
power amplifiers for system design purpose. The problem of finding a good model is
discussed through solutions offered by neural networks, with particular interest in wavelet
networks trained by BFGS algorithms. Finally, experimental results are reported. 䊚 1999
John Wiley & Sons, Inc. Int J RF and Microwave CAE 9: 198᎐215, 1999.

Keywords: telecommunications system; multilayer perceptron; wavelet network; nonlinear

modeling; wavelet transform; simulation

I. INTRODUCTION The same methodology is then applied on cir-

Currently, the most widely used system design
methodology is composed of two stages:
1. Top-down design stage: From the problem
statement, the target system is split up into
subsystems or circuits based on the pro-
posed architecture, the desired details, and
the electrical requirements. The specific sys-
tem is replaced by a hierarchical system-
level model composed of a behavioral cir-
cuit model based on input᎐output relations.
The behavior of this hierarchical model
might be analyzed by system simulators to
improve circuit models specifications.
Correspondence to: Y. Harkouss
cuit models to split up circuits into subcircuits or
devices. The focus of the circuit design is the
technology and the architecture needed to meet
electrical requirements. The circuit-level hierar-
chical model built at this design step is composed
of basic passive or active device models, physical
or electrical. Circuit architecture and device
choices are finally improved by analyzing circuit
performances, by means of circuit simulators.
2. Bottom-up design stage: Component models
which constitute a circuit are extracted from,
for example, heavy simulations by PDE
solvers for active devices, or, more directly,
from measurements. The electrical behav-
iors of the circuits which compose the over-
all system model are then analyzed and
optimized to meet as closely as possible

䊚 1999 John Wiley & Sons, Inc. CCC 1096-4290r99r030198-18

198

specifications established at the first stage.
Next, circuit specifications in the hierarchi-
cal system model are replaced by previously
evaluated circuit performances, and the
overall system behavior is analyzed by means
of a system-level simulator, in order to ver-
ify the target specifications.
Figure 1 shows the two stages of the design
process. In practice the designer goes through
several iterations until he arrives at a stable sys-
tem model that adequately covers the require-
ments.
Accurate information modeling across the hi-
erarchy and between heterogeneous tools that are
being used is one important aspect of this design
process. Going up in the hierarchy implies the use
of specific simulation tools acting on one hand on
specific models, from low-level physical models to
high-level behavioral models, and on the other
hand on specific signals, from one carrier to com-
plex modulations. At each level the product
model-complexity = signal-complexity must be
constant in order to allow simulation in realistic
delays.
The development of more efficient tools for
hierarchical modeling of systems, circuits and de-
vices, and means to embed more information into
the models are the focus of this paper.
Figure 1. Two stage design process.
Figure 2. Device equivalent electrical circuit.
ANN in Microwa¨ e De¨ ice and Circuit Modeling 199
II. DEVICE AND CIRCUIT MODELS
A. Nonlinear Device Model
Adequate modeling of nonlinear microwave de-
vices still remains a complex but unavoidable step
toward the achievement of a good circuit design.
Lumped equivalent electrical circuits, an example
is shown Figure 2, which are today’s mostly used
models w1x, offer the advantage of being computa-
tionally efficient and accurate, but at the expense
of very complex model parameters extraction car-
ried through numerical fitting and optimization
w2x joined to the availability of an accurate circuit
structure.
As an alternative, black box models relying on
directly measured data enable us to get rid of the
circuit topology. The Volterra series approach has
recently re-emerged through the use of time vary-
ing Volterra kernels w3, 4x, in the case of short
memory devices, these series converge rapidly
and thus can be limited to a first order kernel.
The use of time varying kernels enables us to step
over the limitation of the classic Volterra series
to weak nonlinearities and avoids us the complex
and tedious task of measuring higher order ker-
nels. The short memory condition tends to be
satisfied by most electronic devices and therefore
the application of the modified Volterra series
approach to these devices modeling may be
justified.
When the short memory criterion is verified,
the time varying Volterra series has the following
form w4x:
i i Ž t . s Ii0  ¨ 1Ž t . , . . . , ¨ n Ž t .4
n P
q Ý Ý Yi j  ¨ 1Ž t . , . . . , ¨ n Ž t . , ␻ p 4 Vj p e j ␻ p t ,
js1 psyP
i s 1 ⭈⭈⭈ n.
200 Harkouss et al.

The DC term Ii0 is defined by DC device B. Circuit Model

measurements and the time varying kernel Yi j is
For the circuit-level simulation purpose, a mi-
directly related to the measured device bias de-
crowave circuit such as amplifier, oscillator, or
pendent Y-parameters, on all the frequency
mixer is always described as an architecture com-
ranges.
posed of linear and nonlinear device models. At
The number of involved terms and axes in-
system-level simulation, no physical insight is
creases rapidly with the number of ports, conse-
available for this circuit which must only be de-
quently a considerable amount of data measure-
scribed by a nonlinear input᎐output mapping,
ments is needed to define the device behavior,
allowing the handling of telecommunication sys-
making prohibitive the memory cost of the device
tems composed of a large number of circuits
table representation especially when more than
acting on complex signals. The general circuit
one such modeled device are involved in the
block diagram is depicted in Figure 4.
circuit analysis. The inefficiency of the traditional
Each of the electrical variables has the follow-
interpolating schemes in the multidimensional
ing form:
case added to the high cost of the required stor-
age space pushed us to seek a solution providing
a set of continuous and infinitely differentiable x Ž t . s XˆŽ t . e j ␻ 0 t ,
mathematical functions defined by a small set of
parameters, and thus enabling us to considerably where Xˆ is a complex envelope signal Žor modu-
reduce the amount of memory space required to lation. and ␻ 0 the carrier frequency.
define the device behavior. Neural networks which In today’s system simulators, circuits could only
are a good data compression method, afford us be described by input᎐output relations such as
with the ability to move from a discrete data
space to a continuous one and thus can be Iˆ1Ž t . s f Ž Vˆ1Ž t . . ,
considered as an efficient solution to the above
problems. Iˆ2 Ž t . s f Ž Vˆ1Ž t . . .
The global neural network architecture model-
ing the device is composed of 10 neural networks These relations do not take into account interac-
ŽDC currents and four Volterra admittances., as tions between the modeled circuit and circuits
shown in Figure 3. which are connected at model ports. Lack of
Learning bases needed for neural networks simulation accuracy induced by this limitation
training are directly obtain from automatic mea- being critical for most system applications, the
surement setup. Choice criteria for neural net- present work considers a new circuit model ex-
work structures and learning algorithms are pressing input᎐output relations in the following
developed in the next section. way:

Iˆ1Ž t . s f Ž Vˆ1Ž t . , Vˆ2 Ž t . . ,

Iˆ2 Ž t . s f Ž Vˆ1Ž t . , Vˆ2 Ž t . . .

As Iˆ and Vˆ are complex variables, we have in fact

four equations in four variables linking Iˆ and Vˆ
real and imaginary parts. A nonlinear black box
structure based on neural networks similar to that
presented in the previous section and shown in

Figure 3. Volterra-ANN device model. Figure 4. Circuit block diagram.


Figure 5. ANN circuit model.
Figure 5 seems to be very adequate for a success-
ful modeling.
From circuit simulations or from measure-
ments, the circuit being realized, input᎐output
relations may be sampled over a large range of
frequency and voltages presented at input and
output, in order to build a learning base. Choice
criteria for neural network structures and learn-
ing algorithms are developed in the following
section.
III. ARTIFICIAL NEURAL NETWORKS
AND LEARNING SCHEME
A nonlinear black box structure for a static or
dynamic system is a model structure that is pre-
pared to describe virtually any nonlinear behav-
iors. There has been considerable recent interest
in this area with structures based on neural net-
works: multilayer perceptron, radial basis func-
tion neural networks, as well as wavelet neural
networks. From a user’s viewpoint, the key prob-
lem is to find a suitable model structure, within
which a good model is to be found. Fitting a
model within a given structure is then, in most
cases, a lesser problem.
An artificial neural network is considered as a
grossly simplified model of the human brain. It is
a computing system made up of a number of
information processing units or artificial neurons
highly interconnected through a set of links rep-
resented by synaptic weights which are adjusted
during a training process.
Neural networks are widely applied to a variety
of practical problems as signal processing, speech
processing, and control systems w5, 6x, analysis and
optimization of microwave circuits w7x, intermodu-
lation and power analysis w2x, and more. Many
successes have been reported. Recently, artificial
neural networks have become a popular tool in
learning nonlinear maps from discrete data, due
to their ability to learn from data, to generalize
ANN in Microwa¨ e De¨ ice and Circuit Modeling 201
patterns in data, and to approximate any degree
of nonlinearity.
A. Multilayer Perceptron
An important class of neural networks, multilayer
feedforward networks w5, 6x, i.e., multilayer per-
ceptron MLP, is widely used to solve a wide range
of problems among function approximation. This
network consists of a set of neurons arranged in
layers: an input layer, one or more hidden layers,
and an output layer. All neurons in a layer are
connected to all neurons in the adjacent layers
through unidirectional links. The input signal
propagates through the network in a forward
direction on a layer-by-layer basis in order to
compute the output of the network. The MLP is
considered as a universal approximator to a non-
linear multidimensional mapping. It has been the-
oretically proven that a MLP with at least one
hidden layer is able to approximate any complex
nonlinear multidimensional relationship w8, 9x. A
block diagram of a three-layer perceptron is illus-
trated in Figure 6. Where X g R d is the input
Ž x i . vector, Y g R p is the output Ž yo . vector, and
wr sŽ r sso h, h i4. is the synaptic weight from neural s
Ž h or i . of the previous layer to the neuron r
Ž o or h..
The oth Ž1 F o F p . nonlinear input᎐output
mapping performed by the three-layer perceptron
can be written as the following:
M L PŽ X . s ␸ wo h ␸ w h i x i q ␪ h q ␪o .
žÝ žÝ
h i
/ /
A MLP is usually a static network with a
forward direction of signal flow and no feedback
loops, and is usually constructed with sigmoid
neurons wwith the sigmoid function as the activa-
tion function: ␸ Ž x . s tanhŽ xr2. s Ž1 y eyx .r
Figure 6. Structure of a three-layer perceptron.
202 Harkouss et al.
Ž1 q eyx . or ␸ Ž x . s 1rŽ1 q eyx .x and trained
with the backpropagation ŽBP. algorithm. How-
ever, due to its multilayered structure and the
greedy nature of the BP algorithm, the training
process often settles in undesirable local minima
of the error surface and converges too slowly.
B. Radial Basis Function Neural Network
The radial basis function ŽRBF. network w9, 5, 6x,
as an alternative to the MLP, has a simpler
structure Žone hidden layer.. One of the special
basis functions that are commonly used is a
Gaussian kernel function. For the RBF networks,
the input patterns of the training data are usually
used as the candidates of the centers of the radial
basis functions, but the strategy to choose the
radius Žor scale. parameter of the radial basis
functions is not clearly defined.
C. Wavelet Neural Network
The wavelet theory w10, 11x has been found to be
a useful tool of the above described problems.
The wavelet theory is a rapidly developing branch
of mathematics, which has offered very efficient
algorithms for approximating, estimating, analyz-
ing, and compressing nonlinear functions. Re-
cently, due to the similarity between discrete in-
verse wavelet transform and one-hidden-layer
neural networks, the idea of combining both
wavelets and neural networks has been proposed
by Q. Zhang and A. Benveniste w12x.
A wavelet network is an adaptive discretization
of the continuous inverse wavelet transform. It
can also be considered as a one-hidden-layer neu-
ral network with wavelets as activation functions
of its hidden neurons and as a multiple-input-
one-input ŽMISO. wavelet network. The output
layer is one linear neuron. A block diagram of a
wavelet neural network is shown in Figure 7.
Because large dimensional wavelet transforms
lead to a large number of coefficients, it is useful
to introduce some simple ways for building multi-
dimensional wavelets from one dimensional ones.
The most natural ones are the tensor product
form w13x and the radial form. The direct product
wavelet network is not suitable for approximation
of functions of a large number of variables be-
cause it is very expensive to construct a high
dimensional direct product wavelet. There-
fore, the wavelet network is better suited for this
problem.
Given a wavelet function ␺ : R d ª R, the
nonlinear input᎐output mapping of the wavelet
Figure 7. Structure of a wavelet neural network.
neural network can be written as
h h
W NN Ž X . s Ý wi ␺i Ž X . s Ý wi ␺ Ž d i Ž X y Ti .. .
is1 is1
In practice this equation is modified as follows:
h
W NN Ž X . s Ý wi ␺ Ž d i Ž X y Ti .. q V T X q ␪ ,
is1
where ␺ Ž⭈. is the mother radial wavelet function,
wi g R are linear weights, ␪ g R is a bias param-
eter, V g R d is the direct linear connection Ž ¨ i .,
vector X g R d is the input Ž x i . vector of network,
d i g R are dilation parameters, and Ti g R d are
translation vectors for the ith hidden wavelet
Ži.e., neuron.. h is the number of wavelets in the
network. All parameters wi , ¨ i , ␪ , d i , Ti of the
network are to be adapted on the training data.
The linear part of W NN Ž X ., V T X q ␪ , is used in
order to more easily capture linear properties in
regressions.
The number of parameters of a wavelet net-
work is given by
n p s h Ž d q 2 . q d q 1.
In this work, the mother radial wavelet func-
tion ␺ has been chosen as
␺ Ž X . s Ž 5 X 5 2 y d . ey5 X 5
2
r2
,
the so-called ‘‘Inverse of Mexican hat.’’ 5 X 5 de-
notes the Euclidean norm of X and d s dimŽ X ..
D. Constructing Wavelet
Neural Networks
In principle we can initialize the network with a
regular wavelet lattice but many useless wavelets
may be included in the network. A better ap-

proach is the ‘‘stepwise selection by orthogonal-
ization’’ w13x Ži.e., techniques of regressor selec-
tion.. The following is an outline of this proposed
approach:
1. Construct a library of discrete dilated and
translated versions of a given mother wavelet
␺ . This library is constructed according to
the available training data set ⌰N s
Ž X i, y i . g R d = R, i s 1, . . . , N; N - ⬁4 ,
where N is the number of patterns in the
training set ⌰N and Ž X i, y i . is the ith sam-
ple of the inputs and desired output of the
network.
2. Select the best regressors Ži.e., wavelets.
from this library in an iterative fashion and
that best fits the training data ⌰N . During
the selection phase, at each stage, one
wavelet is automatically selected in a direc-
tion to minimize the mean squared error
MSE between the desired output and the
network output,
N
1 1
MSE s Ý Ž y i y W NN Ž X i .. .
N is1 2 any special artificial neural network’s ŽANN’s.
When the regressors are selected, the linear
coefficients wi are determined by the least
squares solutions.
The choice of the number of hidden neurons
Ži.e., number of model parameters. is based on
the complexity of the nonlinearity and on the
requested accuracy for the approximation. For
this type of network, we simply apply the Akaike’s
final prediction error criterion for determining
the number of neurons in the wavelet network.
This criterion is given by: JF P E s Fp .MSE, where
Fp s Ž1 q n prN .rŽ1 y n prN . is the factor of
penalty.
E. Training Algorithms
All above network structures might be trained
using a wide range of algorithms such as the
celebrated backpropagation error algorithm, or
the conjugate gradient and quasi-Newton algo-
rithms. The BP algorithm is very general and not
limited to the one-hidden-layer sigmoid neural
network, but in practice the training process often
settles in undesirable local minima of the error
surface and converges too slowly.
In the case of function approximation, where
the size of the problem Ži.e., the number of pa-
ANN in Microwa¨ e De¨ ice and Circuit Modeling 203
rameters to be determined. is not very large, we
may make use of robust quasi-Newton optimiza-
tion methods such as BFGS ŽBroyden᎐Fletcher᎐
Goldfarb᎐Shanno. or LBFGS Žlimited memory
BFGS. algorithm w14x. In this application, the
BFGS algorithm is clearly superior to the BP
approach and is thus preferred for wavelet neural
networks ŽWNN. Žor MLP. training. Detailed
comparisons show that the BFGS method is typi-
cally 30᎐100 = faster than BP and the LBFGS
method is superior to the BFGS method when the
number of parameters in the networks is large
Ži.e., large scale optimization. and is very competi-
tive due to its low iteration cost w15x.
F. Wavelet Network and Broyden –
Fletcher – Goldfarb – Shanno
The choices that have to be made and considera-
tions that are relevant to do that are the key
points for a successful application of these
techniques.
Modeling tools built on these techniques have
2 to be used by microwave system designers without
background, they must be robust and able to
deliver an accurate model in the shortest time,
with less parameter value choice than possible.
After several experiments using different ar-
chitectures coupled with different training algo-
rithms, we have chosen for our applications to use
a wavelet network coupled with the BFGS opti-
mization algorithm. This choice is the one which
better verifies the previous requirements. There
are several advantages by combining wavelets
and neural networks, for instance, results from
the wavelet theory are a better alternative of the
theorems on the approximation ability of the
feedforward neural networks, and the wavelet
transform provides useful guidelines for the con-
struction of wavelet networks. Therefore, the
wavelet neural networks ŽWNN. and one algo-
rithm for determining the parameters of net-
works, that is the procedure for building WNN
based on training data by combining techniques
in regression analysis and the commonly used
quasi-Newton BFGS procedure, are chosen.
The main key points of this choice are the
following:
䢇 The good initialization of the wavelet net-
work yields a fast training procedure.
䢇 During the training process the wavelet neu-
ral network automatically adjusts its param-
204 Harkouss et al.
eters Ži.e., wi , d i , Ti . so that the error MSE
is minimized. In most cases, the BFGS algo-
rithm leads to the lowest MSE in the short-
est time. However, for large scale applica-
tions, the LBFGS algorithm may favorably
replace BFGS.
The choice of the number of hidden neurons
Ži.e., number of model parameters. is based on
the complexity of the nonlinearity and on the
requested accuracy for the approximation.
IV. MODELING EXAMPLES
In this section, a nonlinear device and circuit
modeling using, respectively, multilayer percep-
tron MLP and wavelet neural network WNN,
which confirm the capacity and the efficiency
of artificial neural networks in the nonlinear
microwave devices and circuits modeling, are
presented.
A. Device Modeling Example
A.1. Volterra Kernel Modeling. In this first exam-
ple, a transistor has been measured, and used in a
13.6 GHz amplifier. The transistor is modeled by
a black box for which signal parameters Žthe drain
to source current Id s , the gate to source current
I g s , and the complex value of each parameter Yi j .
are evaluated through 10 neural networks, based
upon the fitting of both of these parameters to
the corresponding Volterra series kernels over
the whole operational range Ž Vd s g w0, 9 Vx, Vg s g
wy4, 0.8 Vx, and Freq g w1, 86 GHzx. for amplifier
simulations.
The input vector for each Yi j neural network
consists of three parameters: frequency Freq, and
the gate-source and drain-source voltages Vg s
and Vd s .
These networks contain 15 neurons in the hid-
den layer Žwith the logistic function as the activa-
tion function. and one output Ž76 parameters᎐
network.. Each DC neural network is composed
of two inputs: gate-source and drain-source volt-
ages, six neurons in the hidden layer Žwith the
TABLE I. Approximation Results after the Learning Procedure
NN1 NN2 NN3 NN4
Network Ids Ig s Re Y11 Im Y11
MSE*1e q 05 0.66 1.05 0.31 0.24
logistic function as activation function., and one
output Ž25 parameters᎐network.. The output vec-
tor of the block containing the 10 neural net-
works as shown in Figure 3 consists of signal
parameters, Id s , I g s , and Yi j parameters. The 10
neural networks are trained separately with the
modified backpropagation algorithm w6x Žon-line
backpropagation algorithm with momentum. us-
ing our simulator NETLET Žneural wavelet simu-
lator., and acting on 350 measurement points in
the learning base for DC characteristics and 7000
measurement points in the learning base for Y-
parameters. In this learning algorithm, the learn-
ing rate is varied according to whether or not an
iteration decreases the total sum squared error
ŽSSE.:
䢇 if the total error function keeps decreasing
after m consecutive iterations, the learning
rate is multiplied by a factor a ) 1 for next
iterations;
䢇 if the total error function keeps increasing
after n consecutive iterations, the learning
rate is multiplied by a factor b - 1 for next
iterations.
Table I shows the approximation results ob-
tained for the Volterra-ANN device model after
the neural networks learning procedure. As an
example, Figure 8 shows the evolution of SSE s
N)MSE Žsum squared error. as a function of
Figure 8. Evolution of sum squared error SSE during
the ReŽ Y11 .’s neural network learning process.
NN5 NN6 NN7 NN8 NN9 NN10
Re Y12 Im Y12 Re Y21 Im Y21 Re Y22 Im Y22
0.58 0.53 0.67 0.49 0.7 0.42
 ANN in Microwa¨ e De¨ ice and Circuit Modeling 205

epochs obtained after the ReŽ Y11 . neural network
learning process. N is the number of patterns in
the training data set. The optimal learning rate
parameter is about 0.2 and the optimal momen-
tum constant is about 0.9.
A classical lumped equivalent electrical circuit
is deduced from the same measurements, in order
to compare simulation results.
A.2. Simulation Results (Test Results). In this sec-
tion, we present the neural networks obtained
test results for inputs never used in the training
data set.
Analyses of a 13᎐14 GHz amplifier under one-
and two-tones excitations were done, the input
voltage source was swept between y10 and
13.5 dBm, the fundamental frequency for the

Figure 9. Measured and artificial neural network generated DC characteristics. Ža. drain
current vs. drain and gate voltages; Žb. gate current vs. drain and gate voltages.
206 Harkouss et al.

one-tone analysis was chosen to be 13.6 GHz, and
for the two-tone analysis the fundamental fre-
quencies were 13 and 14 GHz. Figures 9 and 10
show some of the characteristics of a TA5446
MESFET from THOMSON foundry, which
lumped equivalent circuit is given in Figure 2, the
curves obtained by the neural model totally agree
with the measurements assuring us of the data
compression scheme efficiency. Plots of the out-
put power vs. input power for the lumped and the
block box models are given in Figure 11. As it can
be seen from the figures, the black box model and
the lumped equivalent circuit are in total agree-
ment, there by proving the efficiency of the fol-
lowed modeling scheme.
The results indicate that the MLP is also able
to generalize and to predict the input᎐output
relationship with high accuracy for input᎐output

Figure 10. Measured and neural network computed Y-parameters against frequency. Ža.
ReŽ Y11 .; Žb. ImŽ Y11 .; Žc. ReŽ Y12 .; Žd. ImŽ Y12 .; Že. ReŽ Y21 .; Žf. ImŽ Y21 .; Žg. ReŽ Y22 .; Žh.
ImŽ Y22 ..

Figure 10. Ž continued..
patterns never used in creating or training the
network.
B. Circuit Modeling Example
B.1. Circuit Modeling. The second example pre-
sents a 2.9 GHz microwave amplifier modeling
directly from measurements. The circuit being
realized, a measurement system, based on active
ANN in Microwa¨ e De¨ ice and Circuit Modeling 207
load pull principle, allows the determination of
input and output variables on a wide range of
feasible physical states. The neural network archi-
tecture is shown Figure 5 and the previous mea-
surements constitute the learning bases of each
network building the overall structure. The learn-
ing base size is about 2625 measurement points.
When the training data set is prepared, the
initialization techniques proposed in Section III.D
208 Harkouss et al.
Figure 10. Ž continued..
are applied to initialize each wavelet neural model
and to determine the number of hidden wavelets
in each network. We simply apply the Akaike’s
final prediction error criterion for determining
the number of hidden neurons in the wavelet
neural model. In Figure 12 this criterion J FPE and
the mean squared error MSE are plotted as a
function of the number of wavelets used in the
ReŽ I1 . network. According to this criterion, 20
wavelets are an appropriate choice. Table II shows
the number of wavelets used in each wavelet
neural network.
The initialized wavelet neural models are
trained separately by the BFGS algorithm and
each wavelet network’s black box model has
been implemented in Cqq-programming language
on a typical UNIX workstation. The determina-
tion of the wavelet neural network parameters

Figure 10. Ž continued..
 d i , Ti , wi , ¨ i , ␪ 4 , during the initialization and the
learning procedures, was done using our simula-
tor NETLET. A few minutes are required to
achieve the wavelet networks training Ž, 5 minr
network..
Table III shows the approximation results ob-
tained for the ANN circuit model after the initial-
ization and the learning procedures. In this table
ANN in Microwa¨ e De¨ ice and Circuit Modeling 209
we listed the mean of square error MSE of all the
networks of the ANN circuit model on the train-
ing data set.
Figure 13 shows the modeling results where
the abscissa represents the load-pull measure-
ment or the training data and the ordinate gives
the approximation or the wavelet neural network
computed output currents. These results demon-
210 Harkouss et al.

Figure 11. Calculated output power of MESFET amplifier. Ža. single-tone harmonic output
power of the 13.6 GHz MESFET amplifier; Žb. two-tone output power Ž f 1 and 2 f 1 y f 2 . of
the 13.6 GHz MESFET amplifier; Žc. two-tone output power Ž f 2 and 2 f 2 y f 1 . of the
13.6 GHz MESFET amplifier.

strate an excellent agreement with the measure-
ment data.
B.2. Simulation Results (Test Results). The output
power of the microwave amplifier under consider-
ation, loaded by a 50 ⍀ impedance, is measured
while sweeping input power. The same character-
istics are computed using the amplifier neural
model previously trained. Results are presented
in Figure 14, and the good agreement between
measured and computed power allow to validate
the approach used.
As it has been detailed in Section II.B, the
main interest of the proposed circuit model is to
be able to take into account interactions between
the modeled circuit and circuits connected at
 ANN in Microwa¨ e De¨ ice and Circuit Modeling 211

Figure 11. Ž continued..

Figure 12. J FPE and MSE as a function of the number of wavelets used in ReŽ I1 .’s neural
network.

TABLE II. Number of Wavelets Used in Each Wavelet TABLE III. Approximation Results after the
Neural Network Initialization and the Learning Procedures
Network Number of Wavelets Network MSEŽinit.. MSEŽfin..
NN1 ŽReŽ I1 .. 20 NN1 ŽReŽ I1 .. 1.03e᎐05 3e᎐07
NN2 ŽImŽ I1 .. 1 NN2 ŽImŽ I1 .. 3.4e᎐15 3.4e᎐15
NN3 ŽReŽ I2 .. 24 NN3 ŽReŽ I2 .. 4.68e᎐05 8.65e᎐07
NN4 ŽImŽ I2 .. 24 NN4 ŽImŽ I2 .. 4.8e᎐05 1.46e᎐06
212 Harkouss et al.

Figure 13. Resulting approximation: measured and wavelet neural network generated
output currents. Abscissa: load-pull measurement, ordinate: WNN model computed. Ža.
ReŽ I1 .; Žb. ImŽ I1 .; Žc. ReŽ I2 .; Žd. ImŽ I2 ..

model ports, due to possible mismatches. Such
mismatches might considerably affect circuit per-
formances as it can be seen in Figure 10 where
two input᎐output power curves are shown for two
different loads and compared to the ideal 50 ⍀
curve. Unfortunately, today’s system simulators
are not able to handle this kind of model and this
limitation leads to a significantly decreasing simu-
lation validity. Present results have been carried
out using our own harmonic balance simulator,
LISA, modified in order to take into account
neural network models.
The results indicate that the WNN is also able
to generalize and to predict the output of network
with high accuracy for inputs never used in the
training set.

Figure 13. Ž continued..
V. CONCLUSION
Artificial neural networks and especially wavelet
neural networks offer a telecommunications sys-
tem designer a powerful tool to carry information
between the different design process stages. At
the device-modeling stage, models based on time
varying Volterra series and ANN free designer
from building equivalent electrical circuits based
ANN in Microwa¨ e De¨ ice and Circuit Modeling 213
on physical insight, are a long and complex task
for new technologies. At the circuit-modeling
stage, ANN based models are able to embed
more information than currently used models and
so to predict more accurately the overall system
behavior. As it has been pointed out, today’s
system simulators cannot take full advantage of
these models so they have to be drastically modi-
fied in order to remove limitations. The design of
214 Harkouss et al.
Figure 14. Harmonic balance simulation of the amplifier loaded by 50 ⍀ and by two
mismatched loads.
a new system simulator is currently one of our
main research topics.
REFERENCES
´ ´ J. Obregon, and J.P. Teyssier, Non linear
1. R. Quere,
characterization and modelling of semiconductor
devices: An integrated approach, 23rd European
Microwave ConferenceᎏWorkshop Proceedings,
Spain, 1993, pp. 18᎐21.
2. J. Rousset, Y. Harkouss, J.M. Collantes, and M.
Campovecchio, An accurate neural network model
of FET for intermodulation and power analysis,
26th European Microwave Conference, Czech Re-
public, 1996.
3. M. Asdente, M.C. Pascussi, and A.M. Ricca, Modi-
fied Volterra᎐Wiener functional method for highly
non linear systems, Alta Frequenza XLV Ž1976.,
pp. 756-312E᎐315E-759.
4. F. Filicori, G. Vannini, and V.A. Monaco, A non-
linear integral model of electron devices for HB
circuit analysis, IEEE Trans Microwave Theory
Tech MTT-40 Ž1992., 1456᎐1465.
5. S. Haykin, Neural networks, a comprehensive foun-
dation, Macmillan College Co. New York, 1994.
6. A. Cichocki and R. Unbehauen, Neural networks
for optimisation and signal processing, John Wiley
& Sons, Inc., and B.G. Teubner, Stuttgart, Ger-
many, 1993.
7. A.H. Zaabab, Q.J. Zhang, and M. Nakhla, A neural
network modeling approach to circuit optimization
and statistical design, IEEE Trans Microwave The-
ory Tech 43 Ž1995., 1349᎐1358.
8. K. Hornik, M. Stinchcombe, and H. White, Multi-
layer feedforward network are universal approxi-
mators, Neural Networks 2 Ž1989., 359᎐366.
9. T. Poggio and F. Girosi, Networks for approxima-
tion and learning, Proc IEEE 78 Ž1990., 1481᎐1497.
10. S. Mallat, Multiresolution approximation and
wavelets orthonormal bases of L2 Ž R ., Trans Amer
Math Soc 315 Ž1989., 69᎐88.
11. C. Chui, Wavelets: A tutorial in theory and applica-
tions, Academic Press, Boston, San Diego, 1992.
12. Q. Zhang and A. Benveniste, Wavelets networks,
IEEE Trans Neural Networks 3 Ž1992., 889᎐898.
13. Q. Zhang, Using wavelet network in non paramet-
ric estimation, IEEE Trans Neural Networks 8
Ž1997., 227᎐236.
14. D.C. Lui and J. Nocedal, On the limited memory
BFGS method for large scale optimization, Math
Programming 45 Ž1989., 503᎐528.
15. S. McLoone and G.W. Irwin, Fast parallel off-line
training of multilayer perceptrons, IEEE Trans
Neural Networks 8 Ž1997., 646᎐653.

BIOGRAPHIES
Youssef Harkouss was born in Beirut,
Lebanon. He received the Engineer
Diploma in E.E. with honors from the
Lebanese University of Beirut, Lebanon,
1993, and the Diploma of D.E.A. in elec-
tronics from the University of Limoges,
France, 1995. In 1994, he joined the
St-Nazaire Hospital for a training period
in medical materials maintenance. Cur-
rently he is working toward the Ph.D. degree in microwave
electronics at IRCOM, University of Limoges, France. His
research interests include advanced neural networks software
development, neural network modeling for microwave devices
and circuits, and CAD of passive devices.
Jean Rousset received the Ph.D. degree in physics from the
University of Limoges, France, in 1976. From 1974 to 1981, he
was an associate professor at the University of Algeria. He
joined the IRCOM in 1982 as a research engineer. He is
currently managing software developments, and his research
interests are in nonlinear analysis methods and simulation
languages ŽPhoto not available..
´ ´ was born in 1971 in
Habib A. F. Chehade
Lebanon. In 1992, he graduated from the
American University of Beirut where he
was studying computer and communica-
tions engineering. In 1993, he received a
specialization degree ŽD.E.S.S.. in
telecommunication networks and systems
engineering from Versailles-St Quentin
University, France, and in 1994 a pre-
Ph.D. degree ŽD.E.A. in microwave electronics from the Uni-
versity of Limoges, France. Since then, he is working on his
Ph.D. in microwave electronics at the IRCOM, Limoges. His
research interests are mainly in microwave device modeling,
CAD development, and telecommunication systems in
general.
ANN in Microwa¨ e De¨ ice and Circuit Modeling 215
Edouard Ngoya received the Dipl.-Eng. Appl. degree from the
Institut des Telecom d’Oran, Algeria, in 1982, and the Ph.D.
degree in electronics from the University of Limoges, France,
in 1988. From 1988 to 1990, he was a research engineer with
Caroline and Racal Redac, where he contributed to the devel-
opment of MIC CAD software. In 1991, he joined the French
Centre National de la Recherche Scientifique ŽCNRS. as a
researcher. His current fields of interests include computer
methods for analysis, modeling, and optimization of mi-
crowave communication circuits. ŽPhoto not available..
Denis Barataud was born in St.-Junien,
France, on October 1970. He graduated
´
from ‘‘Ecole Nationale Superieure de
´´
Telecommunications de Bretagne’’ in
1994. Since 1995, he has been with the
Microwave Laboratory of the University
of Limoges, studying for the doctor de-
gree in electronics engineering. His main
research interest is the time domain
characterization of nonlinear devices.
Jean-Pierre Teyssier received the Mas-
ters degree in electronics and the Ph.D.
degree from the University of Limoges
ŽLimoges, France. in 1988 and 1994, re-
spectively. He has developed, at the Re-
search Institute of Microwave and Opti-
cal Communications ŽIRCOM, Limoges,
France., a pulsed measurement setup de-
voted to the nonlinear characterization of
microwave transistors. In 1995, he was appointed to the Uni-
versity Institute of Technology ŽIUT Brive France.. He works
on the characterization and modeling of nonlinear devices.