Nonlinear Time-Series Analysis: Ulrich Parlitz

8 NONLINEAR TIME-SERIES
ANALYSIS
Ulrich Parlitz
Drittes Physikalisches Institut, Universitat Gottingen

Biirgerstrasse 42-44, D-37073 Gottingen, Germany
ulli@physik3.gwdg.de
Abstract: This tutorial review presents an overview of the achievements and

some present research activities in the field of state space based methods for non-
linear time-series analysis. In particular, questions of state space reconstruction,
of modelling and prediction, of filtering and noise reduction, of detecting non-
linearities in time series, and applications using chaotic synchronization are ad-
dressed. Furthermore, a new approach for modeling data from spatia-temporal
systems is presented.
8.1 INTRODUCTION
Nonlinearity introduces a wealth of genuine physical phenomena. Among them,
chaotic dynamics is the most puzzling and intriguing one. This type of irregular
dynamics calls for new methods to cope with and has already set out for appli-
cations. The investigations on chaotic dynamics have shed a new light on the
notions of predictability and determinism, because the deterministic dynamics
of even simple systems may be notoriously difficult to predict and even impossi-
ble in the long run. Such systems abound in nature, and examples can be drawn
from almost any scientific discipline (see, for instance, [1, 2, 3, 4, 5, 6, 7, 8, 9)).
In mathematical models of dynamical systems the dynamics is described
in their state space, whose (integer) dimension is given by the number of the
dependent variables of the model. In experiments, however, usually just one
variable is measured as a function of time, and the state space is not normally
known. How, then, to arrive at the attractor that may characterize the system?
This gap between the theoretical notions and observable quantities was filled
in 1980 when Packard, Crutchfield, Farmer, and Shaw [10) published their fun-
damental paper "Geometry from a time series" where for the first time state
209
J. A. K. Suykens et al. (eds.), Nonlinear Modeling
© Springer Science+Business Media Dordrecht 1998
210 NONLINEAR MODELING
space reconstruction methods were applied to scalar time series. A mathemat-

ical justification of this approach was given by Takens [11] at about the same
time. He proved that it is possible to (re) construct , from a scalar time series
only, a new state space that is diffeomorphically equivalent to the (in general
unknown) original state space of the experimental system. Based on these re-
constructed states the time series can then be analyzed from the point view
of (deterministic) nonlinear dynamics. It is possible to model and predict the
underlying dynamics and to characterize the dynamics in terms of dimensions
and Lyapunov exponents, for example. In the following we will address some
of these issues. For further reading we refer to other review articles and books
on this topic [12, 13, 14, 15].
8.2 STATE SPACE RECONSTRUCTION

Essentially two methods for reconstructing the state space from scalar time se-
ries are available: delay coordinates and derivative coordinates. Derivative co-
ordinates were used by Packard et al. [10] and consist of higher order derivatives
of the measured time series. Since derivatives are susceptible to noise, deriva-
tive coordinates usually are not very useful for experimental data. Therefore,
we will discuss the method of delay coordinates only.
Let M be a smooth (C 2 ) m-dimensional manifold that constitutes the
(original) state space of the dynamical system under investigation and let
¢/ : M --+ M be the corresponding flow. Suppose that we can measure
some scalar quantity s(t) = h(x(t)) that is given by the measurement func-
tion h : M --+ R, where x(t) = ¢l(x(O)). Then one may construct a delay
coordinates map
F: M --+ Rd
X t--+ Y = F(x) = (s(t), s(t + tl), ... , s(t + (d - l)tl)
that maps a state x from the original state space M to a point y in a recon-
structed state space R d , where d is the embedding dimension and tl gives the
delay time (or lag) used. Figure 8.1 shows a visualization ofthis construction.
Takens [11] proved that for d ~ 2m + 1 it is a generic property of F to be an
embedding of Min Rd, i.e., F: M --+ F(M) C Rd is a (C 2 -) diffeomorphism.
Generic means that the subset of pairs (h, tl) which yield an embedding is
an open and dense subset in the set of all pairs (h, tl). This theorem was
generalized by Sauer, Yorke and Casdagli [16, 17] in two ways. Firstly, they
replaced the condition d ~ 2m + 1 by d > 2do(A) where do(A) denotes the
capacity (or: box-counting) dimension of the attractor A eM. This is a great
progress for experimental systems that possess a low-dimensional attractor
(e.g., do(A) < 5) in a very high-dimensional space (e.g., m = 100). In this case,
the theorem of Takens guarantees only for very large embedding dimensions d
(e.g., d ~ 201) the existence of a diffeomorphic equivalence, whereas with the
condition of Sauer et al. a much smaller d will suffice (e.g., d> 10). The second
improvement of Takens' embedding theorem by Sauer et al. refers to the term
"generic" that was replaced by "prevalent" which essentially means "almost
NONLINEAR TIME-SERIES ANALYSIS 211
attractor in the reconstruction

unknown of the attractor
tate space M in IRd
measurement \ h
h:M IR ~
ob ervable =h(x )
mea ured time eries
Figure 8.1 Delay reconstruction of states from scalar time series.
all (h, tl)" will yield an embedding. This modification of the original theorem
was necessary, because examples of open and dense (=generic) sets have been
found that are rather "thin" [16, 17]. Furthermore, Sauer et al. showed that
for dimension estimation an embedding dimension d > do(A) suffices. In this
case the delay coordinates map F is, in general, not one-to-one, but the points
where trajectories intersect are negligible for dimension calculations. More
details about the reconstruction of states, in particular in the presence of noise,
may be found in Refs. [18, 19].
If the data are measured with a high sampling rate so-called Broomhead-
King-coordinates [20, 21] may be advantageous. With this method a very
high dimensional reconstruction is used and then a new coordinate system
is introduced where the origin is shifted to the center of mass of the recon-
structed states and the axes are given by the (dominant) principal components
of the distribution of points (states). This new coordinate system is based on
a Karhunen-Loeve transformation that may be computed by a singular-value
decomposition. A discussion of the advantages (e.g., noise reduction) and disad-
vantages of this "post-processing" of the reconstructed states may, for example,
be found in Ref. [22].
For time series that consist of a sequence of sharp spikes (e.g. from firing
neurons) delay embedding may lead to very inhomogeneous sets of points in the
reconstructed state space that are difficult to analyse. As an alternative one
may use in this case the time intervals between the spikes as components for (re)
Figure 8.2 Colpitts Oscillator.
constructed state vectors. More details about this state space reconstruction
from interspike intervals may be found in [23, 24, 25].
Another topic that is worth mentioning although we will not discuss it in
more detail in this review are embedding theorems for driven systems [26].
These remarkable results pave the way for extending the methods to be dis-
cussed in the following to input-output systems with deterministic or stochastic
input.
In the following we will assume that the scalar signal s(t) = h(x(t)) is
sampled with a sampling time ts. The resulting time series {sn} with sn =
s(nts) is used to reconstruct the states
(8.1)
for n = 1, ... , N. The symbol l denotes the delay time or lag in units of the
sampling time (i.e., tl = its). As an example we will use experimental data
from a chaotic Colpitts Oscillator [27].
If the internal dynamics of the transistor is neglected this electronic oscillator
can be described by three ordinary differential equation:
C dVCl h - j3g(VC2 )
l~
C dVC2 VE + VC2
2~ - R - h - g(VC2) (8.2)
Ldh
dt
where
Is V
g(V) = t3 exp( Vr )'
In our experimental and numerical implementation of this circuit we use

as parameter values Rl = 20 st, L = 2.7 mH, C1 = 4.7 p,F, C2 = 4.7 p,F,
R = 390 st, (3 = 200, Is = 14 fA, Vr = 26 mY, Vc = 5 V, and VE = -5 V.
8.3 INFLUENCE OF THE RECONSTRUCTION PARAMETERS

The reconstruction using delay coordinates is based on two parameters: the
embedding dimension d and the delay time t/. We shall now discuss the influ-
ence of both parameters on the reconstruction, because this will turn out to be
crucial for any successful subsequent analysis.
8.3.1 Choice of the delay time

Concerning the choice of the delay time t/ the theorems exclude only a few dis-
crete (non-generic) values. Thus, different t/ lead to reconstructions of the at-
tractor that are diffeomorphic ally equivalent but geometrically different. When
the delay time t/ is too small, the coordinates yi = sn+(i-l)l of each recon-
structed state yn do not significantly differ from one another and therefore the
state points are scattered along the diagonal (Fig. 8.3a and Fig. 8.4b). In this
case, any investigation of a possible fractal structure of the attractor (e.g., a
dimension estimation) becomes difficult because one has to zoom into the cloud
of points investigating scales that are (much) smaller than the diameter of the
cloud perpendicular to the diagonal. On very small scales, however, such inves-
tigations become almost impossible due to the limited number of data points
and the unavoidable presence of noise.
. . ... IR.d... .
.-... .......
...
. :.-. .. ..
.. :... ............ ..-.::-.. .
.-
. ...... . . ....... . . .:.

. :- .... . .......-.:
••• •• e.
(b) •• (e)
Figure 8.3 Sketch of three reconstructions with different delay times: (a) delay too small,
(b) delay o. k., (c) delay too large.
The situation improves when tl is increased. Then the attractor unfolds

and its inner structure becomes "visible" already on larger scales as can be
seen in Figs. 8.3b and 8.4c. When the delay time t/ is increased further, the
reconstructed attractor is folded more and more (Fig. 8.3c and Fig. 8.4c). This
folding does not occur for periodic signals and its origin is the "stretch and fold"
(a) . - - - - - -" (b) __------r,
(c)------~ (d) __------r,
Figure 8.4 (a) Chaotic attractor of the Colpitts oscillator (8.2) in the original state space
(VCl' VCl, h). (b )-( d) Three reconstructions of the attractor shown in (a) with different
delay times tl' The two black dots indicate the locations of the same states on the different
attractors.
mechanism that is typical for chaotic systems with sensitive dependence on

initial conditions. Additional folds in the reconstruction are unwanted because
they bring states close together in the reconstructed state space that are not
close together in the original state space. In particular in the presence of noise,
where state points are additionally shifted from the "right places", this may
lead to cases where states are wrongly addressed. Even without noise, a too
large value of tl may lead to a folding that yields intersections of the attract or
where the reconstruction is not one-to-one anymore. In this case, one may
increase the dimension d to "repair" the embedding, but of course it is more
sensible to decrease tl' As an example for this phenomenon we shall use data
sampled from the Henon map
Xn+l 1- ax~ + bYn (8.3)

Yn+l Xn
with a = 1.4 and b = 0.3. Figure 8.5a shows a reconstruction with a time delay
of tl = 1 that provides a perfect unfolding of the underlying attractor.1 The
reconstruction with delay ~ = 2, however , that is shown in Fig. 8.5b was not
successful, because due to additional folding there are "crossings" on the set
of points where the induced flow in the reconstruction space is not one-to-one
anymore. As an example, one of these crossing points is encircled and the
locations of (pre-) images of both branches are given by the + (-) signs.
(a)
o o
- 1
\
Figure 8.5 Reconstruction of data from the Henon map with (a) delay tl = 1 and (b)
delay tl = 2. In the latter case , the reconstruction is not one-to-one anymore .
These examples show that a proper choice of tl is crucial for any further
investigation of the data.
8.3.2 Choice of the embedding dimension

When the dimension d is chosen too small the conditions given in the embedding
theorems are not fulfilled. For d too large, on the other hand, practical problems
occur due to the fixed amount of points, that constitute thinner and thinner
sets in Rd when d is increased. Therefore, the optimal choice is the smallest
value of d that provides a proper reconstruction.
8.3.3 Estimating reconstruction parameters

Several algorithms have been suggested to find "optimal" values for d and tl.
Most of the methods for determining d are based on continuity tests for the
induced flow in the reconstruction space [28, 29, 30, 31, 32, 33, 34] or for the
embedding itself [35, 36, 37, 38]. The main idea is to check whether closely
neighbouring points are mapped to neighbouring points. The methods differ
with respect to their concrete implementation. To find optimal values for tl one
1 Notethat the Henon map possesses the unusual feature that its "original" state space
{(Xn,Yn)} coincides with its delay reconstruction space {(xn , xn-d}.
class of algorithms directly considers the geometry of the reconstructed attrac-

tor, in particular its unfolding from the diagonal [39, 40]. The most popular
approach consists in a minimization of the redundancy of the coordinates of
the reconstructed states using linear autocorrelation functions or information
theoretic concepts like mutual information [41, 42, 43, 44, 45]. Since bad values
for tl may destroy the embedding, the continuity tests for estimating d may
also be used to determine tl [29, 31, 32, 33, 35].
Until now, no method for determining optimal embedding parameters seems to
be accepted as being the most useful. The reason for this diversity probably
is the fact that there exists no pair (d, td that is optimal for all possible sub-
sequent investigations and applications, and there is some evidence that the
window length dtl is more important than the individual values of d and t/2.
FUrthermore, many of the algorithms proposed need as much computation time
as a direct computation of a quantity of interest, e.g., the dimension. In this
case, one may repeat the computation for several values of d and tl and thus
obtain not only "optimal" reconstruction parameters but also a test for the
robustness of the method used and a first (quantitative) information about the
data.
8.4 NOISE REDUCTION

Experimental data usually are contaminated with noise. To improve noisy
data two possibilities are available: Linear filters and special nonlinear noise
reduction methods that make use of the deterministic origin of the data.
8.4.1 Linear Filters

The influence of linear filters on the reconstruction of states was first investi-
gated by Sauer et al. [16, 17] and by Broomhead et al. [46].
FIR filter. For finite impulse response (FIR) filters one can show that those
filters which destroy the embedding constitute linear subspaces in the space of
all FIR-filters. Therefore, with probability one, FIR-filters preserve the embed-
ding. Nevertheless, one may find examples where the application of a FIR-filter
makes subsequent estimations of Lyapunov exponents or dimensions more dif-
ficult as compared with the original "noisy" time series. In these cases the
influence of the FIR filter on the geometry of the reconstructed is essentially
twofold: (i) Due to the filter elements of the time series are connected that
are not close neighbours in time. This leads to (over) folding of the attractor
(similar to a choice of too large a delay time). (ii) On the other hand, the
FIR filter averages the values of the time series and thus the components of
subsequently generated state vectors are more or less similar. This leads to
2 As an example the reader may verify that the embedding of a T-periodic sinusoidal signal
in a d-dimensional reconstruction space is optimal (Le. a circle) for dtl = kT/2 where k may
be any integer that is not a multiple of d.
a concentration of states along the diagonal of the reconstructed state space

(similar to a choice of too small a delay time). Both effects together lead to a
folded set of points along the diagonal in the reconstructed state space that is
difficult to analyse. Therefore, before filtering the data, one should always try
to investigate the raw data.
IIR filter. Infinite impulse response (IIR) filters may also destroy the dif-
feomorphic equivalence between the original state space and the reconstructed
state space. In this case, the original system and the filter may be considered
as a single, extended system, and the output of this system depends very much
on its filter subsystem. To illustrate this problem consider a simple discrete
low-pass filter
U n +l = QUn + Xn (0 < Q < 1)
that is driven by some other discrete dynamical system (e.g. the Henon map
(8.3)). The Lyapunov spectrum of the combined system consists of the Lya-
punov exponents {AI, ... , Am} of the driving system and the exponent A, = In Q
of the filter. By increasing Q -t 1 one may now shift the exponent A, into the
range of the spectrum {AI, ... , Am} and modify in this way the Lyapunov di-
mension 3 of the complete system. If the filter is not sufficiently dissipative it
may thus increase the dimension of the time series. A rigorous mathematical
treatment of this problem may be found in Ref. [47].
8.4.2 Nonlinear signal separation

If it is known that the data originate from a deterministic system, methods
for nonlinear noise reduction may be applied that have been developed only
recently. The basic idea is to exploit the underlying determinism by correcting
the states and the values of the time series using a (local or global) model that
has previously been fitted to the (raw) data. More details about these applica-
tions of nonlinear time series analysis for noise reduction (or signal separation
[48]) can be found in Refs. [49, 50, 51].
8.5 DETECTING NONLINEARITIES

If the time series originates from a yet unknown process it is important to in-
vestigate the question whether the data contain some nonlinear (deterministic)
dependencies or whether it is a purely linear (stochastic) signal, for instance,
coloured noise. Several methods have been divised to answer this question
[52,53,54,55,56,57,58,59,60,61,62,63,64,65]. Here, we want to focus on
the method of surrogate data [52, 53] that provides a rather general and flexible
framework for investigating unknown data. The basic idea is to make some
hypothesis about the data and then to try to falsify this hypothesis. As an
3For a definition of the Lyapunov dimension based of the Kaplan-Yorke conjecture see for
example [1, 3].
example we will use the widely used hypothesis: "The data are (just) coloured
noise generated by a linear stochastic process". In the first step, the data are
modified in such a way that all structure except for the assumed properties will
be destroyed or "randomized". In the case of the above cited nullhypothesis
this may be done, for example, by Fourier transforming the data and substi-
tuting the phases of the complex Fourier transform by random numbers. The
power spectrum (or equivalently, the autocorrelation function) is not affected
by this modification. After transforming back into the time domain we thus
obtain a new time series with the same power spectrum. If the original data
are just coloured noise we do not have destroyed any underlying structure, and
subsequent estimations of dimensions, Lyapunov exponents, prediction errors,
etc. should give the same results for the original time series and the surrogate
data. If, however, the analysis yields significant differences, then our original
time series was more than "just noise". To improve the statistical robustness
of this test one usually generates several surrogate data sets and compares the
mean values of the quantities obtained in the course of the subsequent data
analysis with the corresponding value from the original time series. Further-
more, it is necessary to take into account a possibly performed static nonlinear
transformation of the data that would distort the Gaussian distribution of the
assumed coloured noise (Algorithm II in Ref. [52], see also Ref. [62]). Exam-
ples for applications of the surrogate data method will be given in the following
sections.
8.6 MODELLING AND PREDICTION

Once the states of the underlying dynamical system have been reconstructed
it is possible to study their temporal evolution. Via the delay coordinates map
F the flow q} in the original state space induces a flow 'ljJt = F- 1 . (pt . F in
the reconstructed state space. To model this induced flow, different methods
have been devised (see, for example, [66,67,68,69, 70, 71] and references cited
therein). An important class are local approximations where only states in a
neighbourhood of a given reference point are used to compute the parameters of
a locally valid model. Possible models are, for example, polynomials (including
locally linear approximations) [72] or radial basis functions [73]. Of course, all
methods for modelling the induced flow locally can in principle also be used
as a global ansatz [73]. In general, local methods are more flexible and often
yield better results when used for prediction of the time series. Global meth-
ods, on the other hand, provide model equations in "closed form" that may
yield a better understanding of the underlying dynamics. When the time series
originates from a continuous dynamical system and a global model is desired
one should try to fit a vector field (differential equation) to the dynamics given
by the reconstructed states. More details concerning different modelling tech-
niques, typical problems and an interesting method for "verifying" a derived
model using synchronization may be found in [74]. In any case, a successful
approximation of the induced flow may not only be used for prediction but
also provides a strong evidence for the deterministic nature of the data. Fur-
thermore, good (local) models are a prerequisite for many other methods of
nonlinear time series analysis and nonlinear dynamics.
8.7 FRACTAL DIMENSIONS

The (fractal) dimension of an attractor characterizes its complexity and gives
a lower bound for the number of equations or variables needed for modelling
the underlying dynamical process.
8.7.1 Correlation dimension

From the infinite family of (generalized) dimensions Dq [15) the correlation
dimension D2 introduced by Grassberger and Procaccia [15, 75) is mostly used
in nonlinear time series analysis. This dimension is given by the scaling
of the correlation sum

2 N i-c ..
Cd(r) = (N _ c)(N _ 1 _ c) ~ ~ H(r - Ilyt - y3 11)
that counts the relative number of neighbouring points closer than r (H is

the Heaviside function with H(x) = 1 for x > 1 and zero elsewhere). The
constant c is some correlation length and is used to omit points that are close
neighbours in time [76), and d is the embedding dimension. The correlation
sum approach is a fixed size method, because for a given radius r the number of
pairs with distance smaller than r is counted. As an example, Fig. 8.6 shows the
correlation sum C3 (r) as a function of the radius r of the ball of neighbouring
points on a doubly logarithmic scale for an experimental chaotic time series
(length 8k, sampling frequency 16kHz) from the Colpitts oscillator. The slope
(Fig. 8.6b) in a suitable intermediate range of r gives the fractal dimension of
the chaotic Colpitts attractor. To detect whether the data are distinct from
noise, surrogate data have been derived from the original data set according
to the method of random Fourier phases as discussed in Section 8.5. Their
correlation sum C3 (r) is included in the figure as a dotted curve. It is strongly
distinct from the curve for the original data indicating a clear difference between
the chaotic data and coloured noise. Furthermore, the results for the surrogate
data indicate the lower and upper limit of the scaling region. The error bars
correspond to ±5 standard deviations.
8.7.2 Information dimension

As an alternative to the fixed size approach one may also use a fixed mass
method to estimate the dimension of the attractor as was pointed out by Badii
and Politi [77, 78) and Grassberger [79). In this case the m nearest neighbours
(8) I (b)
0.1
3
..
.
Q.
~ .5! 2
tn
J'10 - 4 III
10- & .s'" I
10- 6
10- 7 o
0 .01 01 0 .01 0.1
r r
Figure 8.6 (a) Correlation sum C3 (r) vs. the distance r of neighbouring points for ex-
perimental data from a chaotic Colpitts Oscillator. (b) Local slope of the curve shown in
(a). The plateau indicates a dimension D2 between 1.8 and 2. The dotted curve is from
surrogate data to detect nonlinearity.
of each reference point yn and the radius rn = r(m) of this cloud of m points
are determined. For the limit m/N --+ 0 one obtains an approximation of the
capacity dimension Do ~ _ logN

I og N1 ",N
L..-n=l r
n
and an estimate of the
logN
information dimension Dl = - N .
-it Ln=l log rn
To investigate the scaling in the limit m/N --+ 0 one can decrease the number
of neighbours m or increase the number of data points N.
Using a fast search algorithm [80] for the nearest neighbour ynn of the re-
constructed state yn one may estimate the information dimension in terms of
the scaling behaviour
of the quantity
(8.4)
for m = 1 and N --+ 00. Figure 8.7 shows an application of this method to the
experimental data from the Colpitts oscillator.
Another very suitable method for dimension analysis is the determination of
the pointwise dimensions and their average [81, 82]. A pointwise dimension D
at a point of an attractor can be calculated by exploring the scaling behaviour
of the "mass" of the attractor in the vicinity of the point, i.e., the number of
I·. I.
.. I · • ·1
-1
q
- 1.5
1000 2000 5000

N
Figure 8.7 Estimate of the information dimension using Eq . (8.4) .
points within a certain distance r. The number N(r) of points in this ball of
radius r should scale like N(r) ex: rD for r --+ o. When this is done for all points
of the attractor, a histogram of pointwise dimensions is obtained. The average
dimension and the standard deviation are numbers then to characterize the
attractor. However, the histogram itself can be considered as a characteristic
of the attractor.
When the attractor is sufficiently low-dimensional, its (fractal) dimension start-
ing from low-dimensional embedding spaces (d = 1,2,3, ... ) will level off at
some value of d and does not increase further. This value then is said to deter-
mine the relevant number of (nonlinear) degrees of freedom, i.e., the number
of dependent variables, of the dynamical system investigated.
More details about dimension estimation methods, their possible pitfalls, ex-
tensions, and many references are given in the review articles [15, 83, 84] and
in [12] .
8.8 LYAPUNOV EXPONENTS

Lyapunov exponents describe the mean exponential increase or decrease of
small perturbations on an attractor and are invariant with respect to diffeo-
morphic changes of the coordinate system. The full set of Lyapunov exponents
constitutes the Lyapunov spectrum which is an ordered set of real numbers
{>.1,A2, ... ,Am},
When the largest Lyapunov exponent Al is positive, the system is said to be
chaotic and to show sensitive dependence on initial conditions. That way, the
notion of chaos can be quantified. For an exact definition and computational
details see, for example, Refs. [85, 86, 87, 88, 89, 90].
The different methods for computing Lyapunov exponents from time series
that have been proposed so far can be devided into two classes: Jacobian-based
methods and direct methods.
8.8.1 Jacobian-based methods

In the Jacobian methods, a model is first fitted to the data and the Jacobian
matrices of the model equations are then used to compute the Lyapunov expo-
nents using standard algorithms which have been developed for the case when
the equations of the dynamical system are known [86,87,88,89]. Usually local
linear approximations are used [91,92,93,94,95,96,97,98,99]. Modifications
and improvements of the basic algorithm include:
• using neighbouring states from a shell (rmin < r < rmax) instead of a ball
(r < rmax) to reduce the influence of noise (see, e.g., Refs. [92,96,97,99])
• using nonlinear approximations of the flow by polynomials [100, 101, 102,

103], radial basis functions [104, 105, 106] or neural networks [107]
• using projections onto linear subspaces based on local Broomhead-King

coordinates [95]
• using the embedding dimension d for selecting the neighbours and a small
local dimension dL for performing the fit and computing the resulting dL
Lyapunovexponents (see, e.g., [92, 101, 102]).
The latter two methods have been devised to avoid so-called spurious Lyapunov
exponents, that occur when the dimension d of the reconstructed state space
is larger than the dimension m of the original state space. Spurious Lyapunov
exponents do not depend on the dynamics but on the approximation scheme,
the noise contained in the data and in some cases on the local curvature of
the attractor. In principle they can take any values. To identify spurious Lya-
punov exponents one can estimate the local thickness of the attractor along the
directions associated with the different Lyapunovexponents [101, 102] or com-
pare the exponents obtained with those computed for the time reversed series
[98,105], because spurious exponents correspond to directions where the attrac-
tor is very thin and because they do not change their signs upon time reversal
(in contrast to the true exponents). The latter method, however, works only
for data of very high quality that enable also a correct estimation of negative
Lyapunovexponents. An investigation of the data requirements for Jacobian-
based methods may be found in Refs. [108, 109].
8.8.2 Direct methods

Direct methods directly estimate the divergent motion of the reconstructed
states without fitting a model to the data. A well-known and widely used
method was introduced by Wolf et al. [90]. There, a reference orbit {yn} is
considered and a neighbouring orbit {zn} starting from the nearest neighbour
zO of yO. After some transient the difference vector uk = zn+k - yn+k points
into the (time dependent) direction corresponding to the largest Lyapunov ex-
ponent ),1. Then lIukll grows, on the average, exponentially with exp(),ltsk)
until it exceeds the range where a linear approximation of the flow at yn holds.
Now the neighbouring orbit {zn} has to be replaced by another neighbouring
orbit that is closer to the reference orbit and whose initial value lies on or
near the line from the current reference state to the last point of the previ-
ous neighbouring orbit in order to preserve the direction corresponding to the
largest Lyapunov exponent. Criteria for the replacement threshold and other
details of the algorithm are given in Ref. [90], including a FORTRAN program.
In principle, it is possible to use this strategy also for computing the second
largest Lyapunov exponent [90], but this turns out to be quite difficult. On the
other hand, it suffices in many cases to establish the existence of at least one
positive Lyapunov exponent. Due to its robustness the Wolf-algorithm is often
used for the analysis of experimental data (see, for example, Refs. [110, 111]).
Another direct method, which is even simpler in terms of less free parameters,
was proposed by Sato et al. [112] and Kurths and Herzel [113]. Very similar to
the Wolf-algorithm, the average exponential growth of the distance of neigh-
bouring orbits is studied on a logarithmic scale, this time via the prediction
error
(8.5)
where ynn is the nearest neighbour of yn. The dependence of the prediction
error p(k) on the number of time steps k may be divided into three phases.
Phase I is the transient where the neighbouring orbit converges to the direction
corresponding to the largest Lyapunov exponent. During phase II the distance
growths exponentially with exp(AItsk) until it exceeds the range of validity of
the linear approximation of the flow around the reference orbit {yn+k}. Then
phase III begins where the distance increases slower than exponentially until it
decreases again due to foldings in the state space. If phase II is sufficiently long,
a linear segment with slope Al appears in the p(k) vs. k diagram. This not only
allows an estimation of the largest Lyapunov exponent Al but also provides a
direct verification of exponential growth of distances to distinguish determin-
istic chaos from stochastic processes where a non-exponential separation of
trajectories occurs [114]. Figure 8.8 gives an example for the determination of
the largest Lyapunov exponent Al of the Colpitts data by this method. The
dotted curve is from surrogate data to detect determinism. The error bars are
for ±5 standard deviations.
Several authors have investigated modifications of this basic algorithm [33, 115,
116]. In particular, those states must be excluded from the search for the near-
est neighbour that are close together in time. Such states belong to the same
segment of the trajectory and thus cannot converge or diverge exponentially. A
possible criterion for a sufficiently large separation in time is the first minimum
or zero of the autocorrelation function of the data set.
1.5 l I I 1 I I I I I I I {I I I I I I I , I
0.5
II III
10 20 30
k
Figure8.8 Prediction error p for experimental Colpitts data vs. the number of time steps
k. The slope of the solid line in the intermediate range of k gives the largest Lyapunov
exponent Al = 0.16.
8.9 SYNCHRONIZATION OF CHAOTIC DYNAMICS

Synchronization of periodic signals is a well-known phenomenon in physics, en-
gineering and many other scientific disciplines. However, even chaotic systems
may be linked in a way such that their chaotic oscillations are synchronized
[117,118,119,120,121,122,123,124]. In particular, the case of one-directional
coupling has been investigated intensely during the last years [125, 126, 127]
because of its potential application in communication systems [129].
8.9.1 Estimating model parameters

Uni-directional coupling is also realized when a measured time series drives
a computer model. If the model is sufficiently accurate and all its parameters
possess the right values one may achieve synchronization between the computer
model and the data. In this way, it is possible the recover also those physical
variables that have not been measured. Note that this enables a reconstruction
of states in the original physical state space and not in a delay embedding space.
Furthermore, not only the variables may be reconstructed using synchronization
but also the parameters of the system. This is done by changing the parameters
of the model until the (average) synchronization error is minimized. A practical
example for this approach may be found in Ref. [128] where the parameters of
a chaotic electronic circuit have been recovered.
8.9.2 Generalized synchronization

If the coupled systems are different from each other more sophisticated types of
(generalized) synchronization of chaotic dynamics may occur where asymptot-
ically for t -+ 00 a function H exists that maps states of the driving system to
those of the driven system [130, 131, 132, 133, 134]. To illustrate the relevance
of this phenomenon for time series analysis we shall consider an example where
a Rossler system
Xl = 2 + Xl(X2 - 4)
X2 -Xl - x3
X3 = x2 + 0.45x3
drives a Lorenz system
Yl = -a(Yl - Y2)
Y2 = ru - Y2 - UY3
Y3 UY2 - bY3
by means of a coupling signal u = Xl + X2 + X3. For this coupled system

generalized synchronization occurs and the state (Yl, Y2, Y3) is asymptotically
given by some (unknown) function of the drive variables. Now let us assume
that we have measured a time series from the drive systems (e.g. the variable
xt) and a time series from the response system (e.g. Yl). Since the response
system "experiences" the dynamics of the drive system it is no surprise that
we can use the response time series to predict the drive time series. This
is in full agreement with Takens' theorem. On the other, we can in general
not expect that the response time series can be predicted on the basis of the
time series from the drive system, because the drive system has no information
about the actual dynamics of the response. This is, however, the point where
synchronization comes into the play. If both systems are in synchrony in the
sense that a functional relationship exists, then we can predict the response time
series from the drive inspite of the uni-directional coupling! For the coupled
Rossler-Lorenz systems given above we find for example that prediction errors
computed using a nearest neighbour approach are practically the same for both
directions. This means that one cannot draw any conclusion about the direction
of the physical coupling from mutual prediction errors!
8.10 SPATIO-TEMPORAL TIME SERIES

The starting point for the analysis methods discussed in the preceding sections
is the measurement of a single observable of the system of interest. Many
interesting dynanlical systems, however, are spatially extended and thus any
description using only a few global observables may be incomplete. On the
other hand technical devices for recording, storing and analysing sufficiently
long spatio-temporal time series (STTS) are widely available now in terms
of video equipment, large disk capacities and fast computers. Therefore, we
present in the following two methods that can be used for dealing with data
from spatially extended systems.
8.10.1 Linear decomposition into spatial modes

Most of the techniques applied so far to STTS are based on decompositions
into spatial modes that constitute orthogonal bases in a very high dimensional
vector space. Examples for such basic modes are the well-know Fourier-modes,
wavelets or bases that are computed from the data for example by means of
a Karhunen-Loeve (KL) transformation (also called Proper Orthogonal De-
composition, Singular Value Decomposition, Empirical Orthogonal Eigenfunc-
tions) [135). The temporal sequence {sn} of spatial patterns or "images" sn
of the STTS is then projected onto this basis and the resulting projection co-
efficients contain all informations about the dynamics. In particular for the
KL-decomposition very often only a few coefficients are significantly different
from zero and therefore suffice for describing the underlying process. In Ref.
[136), for example, this method was successfully applied to experimental data
from a chemical surface reaction. However, not always can a linear decompo-
sition into basic modes yield a low-dimensional description of the data, even
in cases, where the STTS is governed by a low dimensional attractor. If the
temporal evolution consists for example of structures (e.g. solitons) that move
periodically along the spatial axes it is not possible to describe this process
using a few linear modes only. In these cases linear decompositions thus do not
provide the most efficient description 4 [137).
The example that will be presented in the following sections consist of a
STTS that stems from a hyperchaotic system and cannot be described with
a few linear modes only. To demonstrate this feature empirical orthogonal
eigenfunctions (EOF) were computed by subtracting the temporal mean from
the STTS
1 N
rnm = snm - -N.i....J
'"""' snm (m -- 1, ... , M) (8.6)
n=l
and performing a singular value decomposition of the resulting N x M matrix

R = (r~)
R = U . diag(wl, ... , w m ) . V T . (8.7)
The N x M -matrix U and the M x M matrix V are both orthogonal and the
columns of V constitute the EOFs or basic modes. The singular values Wi are
ordered with respect to their size starting with the largest. The squares of
the Wi give the energy (power) of the corresponding mode and are a measure
of the statistical significance of the corresponding basis vector (pattern) for
describing the STTS. If only a few singular values Wi are significantly larger
4Note that this statement is valid not only for modes that are computed using the KL-
transformation, but for any linear decomposition of the STTS.
than the remaining Wi, the STTS can be described by the projection coefficients
with respect to these dominant spatial modes. In this case the images {sn}
constitute N vectors in JRM that span only a low dimensional linear subspace.
The columns of V corresponding to the large singular values constitute a basis
of this subspace. In order to show that such a low dimensional subspace does
(not) exists for a STTS it suffices therefore to plot the distribution of singular
values.
8.10.2 Local models

An alternative to a decomposition into global linear modes is the construction
of local (nonlinear) models. This approach is motivated by the fact that most
physical equations for spatially extended systems are formulated locally, e.g.
like partial differential equations.
Local Reconstruction of States. For reasons of simplicity and clear pre-

sentation we assume in the following that the local dynamics is the same at
different points in space (except for boundaries). This assumption of homogene-
ity will of course not be fulfilled perfectly by real world experimental systems
but may be viewed as a good approximation for many cases. A generalization
for nonhomogeneous systems is possible and will briefly be discussed below.
As a first step the local dynamics is reconstructed by means of a general-
ization of the delay embedding of scalar time series. Based on this particular
reconstruction many methods from standard nonlinear time series analysis can
be applied to model or predict the dynamics of the STTS. In the following we
present as an example numerical prediction results for a complex STTS that
was generated by a coupled map lattice.
Let {sn} be a sequence of snapshots (images) of the spatio-temporal evo-
lution with n = 1, ... , N. Each snapshot sn consists of M elements (pix-
els) and is therefore represented as aM-dimensional vector with elements s~
(m = 1, ... , M). In the following we will discuss only the case of one dimensional
spatial pattern (one spatial degree). However, generalizations for two dimen-
sions (e.g. video images) or even three dimensional sequences (e.g. holographic
movies) are straight forward. The entire STTS is thus given by a N x M-
matrix. The state of the system at position m and time n is reconstructed in
analogy to the delay embedding of scalar time series. Here we use the center
element s~, some of its neighbours and the corresponding values in the past to
construct the state vector
x
nm
= (n n n n-JL n-JL
sm-IK,···,sm,···,sm+IK,···,sm_IK,···,sm
n-JL )
, ... ,sm+IK
where I is the number of spatial neighbours, J is the number of temporal
neighbours (in the past), K is the spatial shift which has a similar meaning
as the time delay L (time lag) known from the delay embedding of scalar
time series. The dimension d of the state vector xnm E R d is given by d =
(J + 1)(1 + 21). This dimension can be reduced if the STTS stems from a
dynamical system that possesses additional spatial symmetries. For the one
dimensional case we may for example assume that the system is invariant with
respect to spatial reflection. The general form of the state vector may then be
written as
X nm = (sn m' g(sn
m-K, snm+K ) , ... , g(sn m+IK ) , ... , Sn-JL
m-IK' sn m , ... ,
( n-JL n-JL))
... , 9 Sm-IK' Sm+IK
with
= g(V, u).
g(u, v)
Possible choices for 9 are for example g(u,v) = .;uv or g(u,v) = ut vthat is
used in following.
The dimension d = (J + 1)(1 + 1) of the symmetrically reconstructed states
xnm E Rd is approximately about a factor of two smaller. For time series
with two or three spatial degrees of freedom this dimension reduction due to
symmetries is even larger (and thus more important).
Spatial boundaries of the experiment or of the STTS can be taken into
account by constructing their states separately but in a way analogous to that
described above. In principle all states xnm containing boundary values of the
STTS have to be divided into different classes where each class is characterized
by the distance of the center index m from the boundary. For predicting or
modelling only states of the relevant class have then to be compared. This
decomposition into classes can easily be implemented by extending the STTS
in its spatial direction with numbers -c, -2c, -3c, ... to the left and c, 2c, 3c
to the right. The parameter c has to be chosen much larger than the largest
value of the STTS. Using this construction all states of a given class are located
in different subspaces of the reconstruction space Rd. Of course, this method
is not the most efficient implementation but it allows to introduce boundaries
without changing the main part of the algorithm. The effective dimension of the
reconstructed boundary states is lower than d. Therefore, in principle different
values of the reconstruction parameters I, J, K and L could (and should) be
used for the boundary to improve the reconstruction (and thus the results of
any subsequent application of it). However, to keep the presentation of methods
and results as comprehensible as possible we will use in the following the same
reconstruction parameters inside the STTS and at its boundaries where the
reconstruction method explained above is used.
Nonlinear Prediction. As an application of the local state reconstruction

we consider here the prediction xnm t-7 S~+T of the STTS where T gives the
prediction time interval. For this purpose firstly a training set of states A =
{xnm} is derived from Ntrain successive images sn of the STTS. For these states
the preimage-image relation xnm t-7 S~+T is known and represents a nonlinear
map f : R d -+ R provided the reconstruction is one-to-one.
For determining the future value S~+T of an element s~ of the STTS the
corresponding state xnm is reconstructed. Then 2d nearest neighbours of xnm
are selected from the training set A and the underlying map f is locally inter-
polated using a general least squares fit
2d
F(x) =L Ck<Pk (x) (B.B)
k=l
where the functions <Pk are radial basis functions <Pk(X) = Jr 2 + Ilx nkmk - xll 2
centered at the 2d neigbouring states x nkmk • The coefficients Ck are computed
from the known future values S~kk+T (corresponding to the neighboring states)
by solving a linear set of equations. Since F is a local approximation of the
dynamics f the prediction of the desired future value S~+T of the STTS is given
by p~+T = F(xnm).
For monitoring the accuracy of this forecasting scheme the prediction error
En(r) at time n
(B.9)
and the average prediction error E(r)
N
E(r) = ~L En(r) (B.I0)
n=l
may be computed. Of course, locally constant or locally linear maps may also
be used to approximate the dynamics f. Furthermore, predictions over longer
periods of time (r > 1) can be computed iteratively by concatenating steps
with r = 1 or as a single step.
8.10.3 Numerical Example

A .STTS was generated using a lattice of diffusively coupled logistic maps
(B.11)
with
f(s) = as(1 - s).
and m = 2, ... , M - 1 and n = 1, ... , N. The boundary conditions were S1 =
0.5 = sM for all times n = 1, ... , N.
Since the structure of the dynamical equation (B.11) is exactly the same as
that of the modelling ansatz it may be no surprise that for this example very
accurate predictions are possible. Table B.l shows the average prediction error
E(r) vs. the number of temporal (I) and spatial (J) neighbours using the
symmetrical reconstruction scheme with L = 1, K = 1 and r = 1. As expected
there is a pronounced minimum for I = 0 and J = 1.
Table 8.1 Average prediction error E(r) (Eq. (8.10)) as a function of the number of
spatial neighbours I (rows) and temporal neighbours J in the past (columns) for K = 1 =
L.
0 1 2 3
0 1.219443 0.000001 0.000014 0.000053
1 0.218676 0.000003 0.000031 0.000102
2 0.048164 0.000007 0.000050 0.000136
3 0.028297 0.000015 0.000083 0.000206
4 0.024331 0.000025 0.000128 0.000353
5 0.024256 0.000049 0.000211 0.000654
The spatio-temporal evolution of the training set is shown in Fig. 8.9a. Fig-
ure 8.9b shows the following 30 steps of the STTS that have been forecasted
using iterated predictions. The resulting predicted STTS is given in Fig. 8.9c
and Fig. 8.9d shows the difference between the original data (Fig. 8.9b) and
the predicted values (Fig. 8.9d). In this case we used the predicted values p~+l
to replace the original elements s~+1 after each step and the prediction error
En was found to grow exponentially.
Although this example may be viewed as trivial it may serve as a motiva-
tion for the approach and demonstrates dearly the power of the local method.
For describing and predicting this STTS using global mode decomposition like
EOFs many modes would be necessary as can be seen from the distribution of
squared singular values shown in Fig. 8.10.
If the general (nonsymmetrical) reconstruction is used instead of the sym-
metric embedding all prediction errors are slightly larger. The difference, how-
ever, is not large due to the low dimension of the reconstruction in both cases.
The numerical examples show that the local reconstruction of states is a pow-
erful method for predicting spatio-temporal time series. It may also serve as
a starting point for deriving a mathematical model of the underlying dynam-
ics and subsequent bifurcation analysis. The scheme discussed in this paper
may be generalized in different directions. If the process generating the STTS
is not spatially homogeneous one may simply add to the dimension d of the
reconstruction space the number ds of spatial dimensions of the problem (Le.
ds = 1,2 or 3) and work then in the enlarged d + ds dimensional space. Of
course, in that case longer training sets will be necessary in general. Another
modification of the scheme proposed concerns the selection of spatio-temporal
neighbours of the center element s~ for reconstructing the local state xnm. At
this point one may take into account that any physical information spreads
with some maximum speed. Instead of using a rectangular region of the matrix
S a triangle (Le. a "light cone") may be more efficient for reconstructing local
states. This feature of locality may also be viewed as a motivation for the
concept of local states and is probably important for a rigorous mathematical
(a) 100
1.0
09
60 06
07
60 0.6
QI
E 0.6
E=
40 04
0.3
20 0.2
0.1
0 0.0
0 20 40 60 60 100
Space
(b) 130
1.0
125 0.6
Q) 120 0.6
E 115 0.4
E= 110
0.2
105
100 00
0 20 40 60 60 100
Space
(C) 130
10
125 0.6
Q) 120 0.6
E 115 04
E= 110
02
105
100 00
0 20 40 60 60 100
Space
(d) 130
10
125 0.6
Q) 120 0.6
E 115 04
E= 110
0.2
105
100 0.0
0 20 40 60 60 100
Space
Figure 8.9 Spatio-temporal time series generated by the coupled logistic maps with M=
100. (a) Training set of length N = 100. (b) Original time series to be predicted. (c)
200
180
160
140
120
100
80
60
40
20
0
5 10 15 20 25 30 35 40 45
Mode
Figure 8.10 Squared singular value spectrum of the STTS shown in Fig. a.ga.
justification of the method in future. Another promising generalization of the

present scheme could be a combination with standard modal decomposition
techniques. The Karhunen-LoEwe transformation may for example be applied
to the large number of local patterns of the time series given by the above
mentioned rectangular regions used for reconstructing the states. If only a few
typical or dominating local modes occur the projection coefficients with respect
to these basic pattern can be used to define local states. The dimension of
the corresponding state space would then be smaller than for the direct recon-
structing discussed in this paper. FUrthermore, such projections may also be
used for noise reduction. These modifications will be discussed in detail else-
where.
Acknowledgments
The author thanks the organizers of the "International Workshop on Ad-
vanced Black-Box Techniques for Nonlinear Modelling", Leuven, July 8-10,
1998 for their kind invitation and support. Thanks go also to my coworkers
for fruitful discussion and exchange of ideas, in particular to the Nonlinear
Dynamics Group of the Drittes Physikalisches Institut, Gottingen. Support by
the German Ministry for Science and Education (BMBF) is gratefully acknowl-
edged.
References
[1] Ott, E. (1993). Chaos in Dynamical Systems. Cambridge Cambridge, University

Press.
[2] Thompson, J. M. T., H. B. Stewart (1986). Nonlinear Dynamics and Chaos.

Chichester, Wiley.
[3] Schuster, H. G. (1988). Deterministic Chaos. 2nd ed., Weinheim, VHC Pub-
lishers.
[4] Berge, P., Y. Pomeau, C. Vidal (1984). Order within Chaos: Towards a Deter-
ministic Approach to Turbulence. New York, John Wiley and Sons.
[5] Moon, F. C. (1992). Chaotic and Fractal Dynamics. New York, John Wiley and
Sons.
[6] Gaponov-Grekhov, A. V., M. I. Rabinovich (1992). Nonlinearities in Action -
Oscillations, Chaos, Order, Fractals. Berlin, Springer.
[7] Lauterborn, W., J. Holzfuss (1991). Acoustic chaos. Int. J. Bifurcation and
Chaos, 1, pp.13-26.
[8] Lauterborn, W., T. Kurz, U. Parlitz (1997). Experimental Nonlinear Physics.
Int. J. Bifurcation and Chaos, 7, pp.2003-2033.
[9] Lauterborn, W., U. Parlitz (1988). Methods of chaos physics and their appli-
cation to acoustics. J. Acoust. Soc. Am., 84, pp.1975-1993.
[10] Packard, N.H., J.P. Crutchfield, J.D. Farmer, R.S. Shaw (1980). Geometry from
a time series. Phys. Rev. Lett., 45, pp.712-716.
[11] Takens, F. (1981). Detecting strange attractors in turbulence. in Dynamical
Systems and Turbulence, eds. Rand, D.A. & Young, L.-S. , Berlin, Springer,
pp.366-381.
[12] Kantz, H., & T. Schreiber (1997). Nonlinear Time Series Analysis. Cambridge
University Press, Cambridge.
[13] Abarbanel, H.D.1. (1996). Analysis of Observed Chaotic Data. Springer, New
York.
[14] Abarbanel, H.D.I., Brown, R., Sidorowich, J.J., & Tsimring, L.S. (1993). The
analysis of observed chaotic data in physical systems. Rev. Mod. Phys., 65(4),
pp.1331-1392.
[15] Grassberger, P., Schreiber, T. & Schaffrath, C. (1991). Nonlinear time sequence
analysis. Int. J. Bif. Chaos, 1(3), pp.521-547.
[16] Sauer, T., Y. Yorke, M. Casdagli (1991). Embedology. J. Stat. Phys., 65, pp.579-
616.
[17] Sauer, T., J.A. Yorke (1993). How many delay coordinates do you need?
Int. J. Bifurcation and Chaos, 3, pp. 737-744.
[18] Casdagli, M., S. Eubank, J.D. Farmer, J. Gibson (1991). State space recon-
struction in the presence of noise. Physica D, 51, pp.52-98.
[19] Gibson, J.F., J.D. Farmer, M. Casdagli, S. Eubank (1992). An analytic ap-
proach to practical state space reconstruction. Physica D, 57, pp.I-30.
[20] Broomhead, D.S., G.P. King (1986). Extracting qualitative dynamics from ex-
perimental data. Physica D, 20, pp.217-236.
[21] Landa, P.S., M.G. Rosenblum (1991). Time series analysis for system identifi-
cation and diagnostics. Physica D, 48, pp.232-254.
[22] Palus, M., I. Dvorak (1992). Singular-value decomposition in attractor recon-
struction: pitfalls and precautions. Physica D, 55, pp.221-234.
[23] Sauer, T.(1994). Reconstruction of dynamical systems from interspike intervals.
Phys. Rev. Lett., 72, pp.3811-3814.
[24] Castro, R., T. Sauer (1997). Correlation dimension of attractors through inter-
spike intervals. Phys. Rev. E, 55(1), pp.287-290.
[25] Racicot, D.M., A. Longtin (1997). Interspike interval attractors from chaotically
driven neuron models. Physica D, 104, pp.184-204.
[26] Stark, J., D.S. Broomhead, M.E. Davies, J. Huke (1996). Takens embedding
theorems for forced and stochastic systems. Proceedings of the 2nd World
Congress of Nonlinear Analysts, Athens, Greece.
[27] Kennedy, M.P. (1994). Chaos in the Colpitts oscilator. IEEE 7rans. Circuits
Syst., 41(11), pp.771-774.
[28] Cenys, A., K. Pyragas (1988). Estimation of the number of degrees of freedom
from chaotic time series. Phys. Lett. A, 129, pp.227-230.
[29] Buzug, Th., T. Reimers, G. Pfister (1990). Optimal reconstruction of strange
Attractors from purely geometrical arguments. Europhys. Lett., 13, pp.605-610.
[30] Alecsic, Z. (1991). Estimating the embedding dimension. Physica D, 52, pp.362-
368.
[31] Buzug, Th., G. Pfister (1992). Optimal delay time and embedding dimension
for delay-time coordinates by analysis of the global static and local dynamical
behavior of strange attractors. Phys. Rev. A, 45, pp.7073-7084.
[32] Gao, J., Z. Zheng (1993). Local exponential divergence plot and optimal em-
bedding of a chaotic time series. Phys. Lett. A, 181, pp.153-158.
[33] Gao, J., Z. Zheng (1994). Direct dynamical test for deterministic chaos and
optimal embedding of a chaotic time series. Phys. Rev. E, 49, pp.3807-3814.
[34] Huerta, R., C. Santa Cruz, J.R. Dorronsore, V. Lopez (1995). Local state-space
reconstruction using averaged scalar products of dynamical-system flow vectors.
Europhys. Lett., 29, pp.13-18.
[35] Liebert, W., K. Pawelzik, H.G. Schuster (1991). Optimal embeddings of chaotic
attractors from topological considerations. Europhys. Lett., 14, pp.521-526.
[36] Kennel, M.B., R. Brown, H.D.1. Abarbanel (1992). Determining embedding di-
mension for phase-space reconstruction using a geometrical construction. Phys.
Rev. A, 45, pp.3403-3411.
[37] Fredkin, D.R., J.A. Rice (1995). Method of false nearest neigbors: a cautionary
note. Phys. Rev. E, 51(4), pp. 2950-2954.
[38] Cao, L. (1997). Practical method for determining the minimum embedding
dimension of a scalar time series. Physica D, 110, pp. 43-50.
[39] Kember, G., A.C. Fowler (1993). A correlation function for choosing time delays
in phase portrait reconstructions. Phys. Lett. A, 179, pp.72-80.
[40] Rosenstein, M.T., J.J. Collins, C.J. De Luca (1994). Reconstruction expansion
as a geometry-based framework for choosing proper delay times. Physica D, 73,
pp.82-98.
[41] Frazer, A.M., H.L. Swinney (1986). Independent coordinates in strange attrac-
tors from mutual information. Phys. Rev. A, 33, pp.1134-1140.
[42] Frazer, A.M. (1989). Reconstructing attractors from scalar time series: a com-
parison of singular system and redundancy criteria. Physica D, 34, pp.391-404.
[43] Frazer, A.M. (1989). Information and entropy in strange attractors. IEEE
Trans. Info. Theory, 35, pp.245-262.
[44] Liebert, W., H.G. Schuster (1989). Proper choice of the time delay for the
analysis of chaotic time series. Phys. Lett. A, 142, pp.l07-111.
[45] Martinerie, J.M., A.M. Albano, A.1. Mees, P.E. Rapp (1992). Mutual informa-
tion, strange attractors, and the optimal estimation of dimension. Phys. Rev.
A, 45, pp.7058-7064.
[46] Broomhead, D.S., J.P. Huke, M.R Muldoon (1992). Linear filters and nonlinear
systems. J. Roy. Stat. Soc., B54, pp.373-382.
[47] Davies, M.E., & K.M. Campbell (1996). Linear recursive filters and nonlinear
dynamics. Nonlinearity, 9, pp.487-499.
[48] Kaplan, D., T. Schreiber (1996). Signal separation by nonlinear projections:
The fetal electrocardiogram. Phys. Rev. E, 53(5), pp.R4326-R4329.
[49] Grassberger, P., R. Hegger, H. Kantz, C. Schaffrath, T. Schreiber (1993). On
noise reduction methods for chaotic data. CHAOS, 3, pp.127-141.
[50] Kantz, H., T. Schreiber, I. Hoffmann, T. Buzug, G. Pfister, C.G. Flepp, J. Si-
monet, R Badii, E. Brun (1993). Nonlinear noise reduction: A case study on
experimental data. Phys. Rev. E, 48, pp.1529-1538.
[51] Kostelich, E.J.,T. Schreiber (1993). Noise reduction in chaotic time-series data:
A survey of common methods. Phys. Rev. E, 48, pp.1752-1763.
[52] Theiler, J., B. Galdrikian, A. Longtin, S. Eubank, J.D. Farmer (1992). Using
surrogate data to detect nonlinearity in time series. In: Nonlinear Modeling
and Forecasting, eds. M. Casdagli and S. Eubank, SFI Studies in the Sciences
of Complexity, Vol.XII (Reading, MA,Addison-Wesley), pp.163-188.
[53] Theiler, J., S. Eubank, A. Longtin, B. Galdrikian, J.D. Farmer (1992). Testing
for nonlinearity in time series: the method of surrogate data. Physica D, 58,
pp.77-94.
[54] Provenzale, A., L.A. Smith, R Vio, G. Murante (1992). Distinguishing between
low-dimensional dynamics and randomness in measured time series. Physica D,
58, pp.31-49.
[55] Smith, L. (1992). Identification and prediction of low dimensional dynamics.
Physica D, 58, pp.50-76.
[56] Takens, F. (1993). Detecting nonlinearities in stationary time series. Int. J. of
Bifurcation and Chaos, 3, pp.241-256.
[57] Wayland, R, D. Bromley, D. Pickett, A. Passamante (1993). Recognizing de-
terminism in a time series. Phys. Rev. Lett., 70, pp.580-582.
[58] Palus, M., V. Albrecht,1. Dvorak (1993). Information theoretic test for nonlin-
earity in time series. Phys. Lett. A, 175, pp.203-209.
[59] Kaplan, D. (1994). Exceptional events as evidence for determinism. Physica D,
73, pp.38-48.
[60] Salvino, L.W., R Cawley (1994). Smoothness implies determinism: a method
to detect it in time series. Phys. Rev. Lett., 73, pp.l091-1094.
[61] Savit, RM. Green (1991). Time series and dependent variables. Physica D, 50,
pp.95-116.
[62] Rapp, P.E., A.M. Albano, I.D. Zimmerman, M.A. Jimenez-Moltaiio (1994).
Phase-randomized surrogates can produce spurious identifications of non-
random structure. Phys. Lett. A, 192, pp.27-33.
[63] Theiler, J. (1995). On the evidence for low-dimensional chaos in an epileptic

electroencephalogram. Phys. Lett. A, 196, pp.335-341.
[64] Schreiber, T., A. Schmitz (1996). Improved surrogate data for nonlinearity
tests. Phys. Rev. Lett., 77(4), pp.635-638.
[65] Schreiber. T (1998). Constrained randomization of time series data. Phys. Rev.
Lett., 80(10), pp.2105-2108.
[66] Judd, K., A. Mees (1995). On selecting models for nonlinear time series. Physica
D, 82, pp. 426-444.
[67] Aguirre, L.A., S.A. Billings (1995). Identification of models for chaotic systems
from noisy data: implications for performance and nonlinear filtering. Physica
D, 85, pp. 239-258.
[68] Aguirre, L.A., E.M.A.M. Mendes (1996). Global nonlinear polynomial models:
structure, term clustering and fixed points. Int. J. Bifurc. Chaos, 6(2), pp.279-
294.
[69] Allie, S., A. Mees, K. Judd, D. Watson (1997). Reconstructing noisy dynamical
systems by triangulation. Phys. Rev. E, 55(1), pp. 87-93.
[70] Szpiro, G.G. (1997). Forecasting chaotic time series with genetic algorithms.
Phys. Rev. E, 55(3), pp. 2557-2568.
[71] Jaeger, L., H. Kantz (1997). Effective deterministic models for chaotic dynamics
perturbed by noise. Phys. Rev. E, 55(5), pp. 5234-5247.
[72] Farmer, J.D., J.J. Sidorowich (1987). Predicting chaotic time series. Phys. Rev.
Lett., 59, pp.845-848.
[73] Casdagli, M. (1989). Nonlinear Prediction of chaotic time series. Physica D, 35,
pp.335-356.
[74] Brown, R. N.F. Rulkov, E.R. Tracy (1994). Modeling and synchronizing chaotic
systems from time-series data. Phys. Rev. E, 49, pp.3784-3800.
[75] Grassberger, P., I. Pro caccia (1983). On the characterization of strange attrac-
tors. Phys. Rev. Lett., 50, pp.346-349.
[76] Theiler, J. (1986). Spurious dimension from correlation algorithms applied to
limited time-series data. Phys. Rev. A, 34, pp.2427-2431.
[77] Badii, R., A. Politi (1984). Hausdorff dimension and uniformity factor of strange
attractors. Phys. Rev. Lett, 52, pp.1661-1664.
[78] Badii, R., A. Politi (1985). Statistical description of chaotic attractors. J. Stat.
Phys., 40, pp.725-750.
[79] Grassberger, P. (1985). Generalizations of the Hausdorff dimension of fractal
measures. Phys. Lett. A, 107, pp.101-105.
[80] Schreiber, T. (1995). Efficient neighbor searching in nonlinear times series anal-
ysis. Int. J. Bifurcation and Chaos, 5, pp.349-358.
[81] Holzfuss, J., G. Mayer-Kress (1986). An approach to error-estimation in the
application of dimension algorithms. in [82], pp.114-122.
[82] Mayer-Kress, G. (ed.) (1986). Dimensions and Entropies in Chaotic Systems -
Quantification of Complex Behavior. Berlin, Springer.
[83] Theiler, J. (1990). Estimating fractal dimension. J. Opt. Soc. Am. A, 7, pp.1055-
1073.
[84) Broggi, G. (1988). Evaluation of dimensions and entropies of chaotic systems.

J. Opt. Soc. Am. B, 5, pp.1020-1028.
[85) Oseledec, V.1. (1968). A multiplicative ergodic theorem. Lyapunov characteris-
tic numbers for dynamical systems. TI-ans. Moscow Math. Soc., 19, pp.197-231.
[86) Benettin, G., L. Galgani, A. Giorgilli, J.-M. Strelcyn (1980). Lyapunov charac-
teristic exponents for smooth dynamical systems and for hamiltonian systems;
a method for computing all of them. Part II: Numerical application. Meccanica,
15, pp.21-30.
[87) Shimada, I., T. Nagashima (1979). A numerical approach to ergodic problems
of dissipative dynamical systems. Prog. Theor. Phys., 61, pp.1605-1616.
[88) Eckmann, J.-P., D. Ruelle (1985). Ergodic theory of chaos and strange attrac-
tors. Rev. Mod. Phys., 57, pp.617-656.
[89) Geist, K., U. Parlitz, W. Lauterborn (1990). Comparison of Different Methods
for Computing Lyapunov Exponents. Prog. Theor. Phys., 83, pp.875-893.
[90) Wolf, A., J.B. Swift, L. Swinney, J.A. Vastano (1985). Determining Lyapunov
exponents from a time series. Physica D, 16, pp.285-317.
[91) Sano, M., Y. Sawada (1985). Measurement of the Lyapunov spectrum from a
chaotic time series. Phys. Rev. Lett., 55, pp.1082-1085.
[92) Eckmann, J.-P., S.O. Kamphorst, D. Ruelle, S. Ciliberto (1986). Lyapunov
exponents from time series. Phys. Rev. A, 34, pp.4971-4979.
[93) Stoop, R., P.F. Meier (1988). Evaluation of Lyapunov exponents and scaling
functions from time series. J. Opt. Soc. Am. B, 5, pp.1037-1045.
[94) Holzfuss, J., W. Lauterborn (1989). Liapunov exponents from a time series of
acoustic chaos. Phys. Rev. A, 39, pp.2146-2152.
[95) Stoop, R., J. Parisi (1991). Calculation of Lyapunov exponents avoiding spuri-
ous elements. Physica D, 50, pp.89-94.
[96) Zeng, X., R. Eykholt, R.A. Pielke (1991). Estimating the Lyapunov-exponent
spectrum from short time series of low precision. Phys. Rev. Lett., 66, pp.3229-
3232.
[97] Zeng, X., R.A. Pielke, R. Eykholt (1992). Extracting Lyapunov exponents from
short time series of low precision. Modern Phys. Lett. B, 6, pp.55-75.
(98) Parlitz, U. (1993). Lyapunov exponents from Chua's circuit. J. Circuits, Sys-
tems and Computers, 3,pp.507-523.
[99] Kruel, Th.M., M. Eiswirth, F.W. Schneider (1993). Computation of Lyapunov
spectra: Effect of interactive noise and application to a chemical oscillator.
Physica D, 63, pp.117-137.
[100] Briggs, K. (1990). An improved method for estimating Liapunov exponents of
chaotic time series. Phys. Lett. A, 151, pp.27-32.
[101] Bryant, P., R. Brown, H.D.1. Abarbanel (1990). Lyapunov exponents from ob-
served time series. Phys. Rev. Lett., 65, pp.1523-1526.
[102] Brown, R., P. Bryant, H.D.1. Abarbanel (1991). Computing the Lyapunov spec-
trum of a dynamical system from an observed time series. Phys. Rev. A, 43,
pp.2787-2806.
[103] Abarbanel, H.D.I., R. Brown, M.B. Kennel (1991). Lyapunov exponents in

chaotic systems: their importance and their evaluation using observed data.
Int. J. Mod. Phys. B, 5, pp.1347-1375.
[104] Holzfuss, J, U. Parlitz (1991). Lyapunov exponents from time series. Proceedings
of the Conference Lyapunov Exponents, Oberwolfach 1990, eds. L. Arnold, H.
Crauel, J.-P. Eckmann, in: Lecture Notes in Mathematics, Springer Verlag.
[105] Parlitz, U. (1992). Identification of true and spurios Lyapunov exponents from
time series. Int. J. Bifurcation and Chaos, 2, pp.155-165.
[106] Kadtke, J.B., J. Brush, J. Holzfuss (1993). Global dynamical equations and
Lyapunov exponents from noisy chaotic time series. Int. J. Bifv.rcation Chaos,
3, pp.607-616.
[107] Gencay, R., W.D. Dechert (1992). An algorithm for the n Lyapunov exponents
of an n-dimensional unknown dynamical system. Physica D, 59, pp.142-157.
[108] Eckmann, J.-P., D. Ruelle (1992). Fundamental limitations for estimating
dimensions and Lyapunov exponents in dynamical systems. Physica D, 56,
pp.185-187.
[109] Ellner, S., A.R. Gallant, D. McCaffrey, D. Nychka (1991). Convergence rates
and data requirements for Jacobian-based estimates of Lyapunov exponents
from data. Phys. Lett. A, 153,pp.357-363.
[110] Fell, J., J. Raschke, P. Beckmann (1993). Deterministic chaos and the first pos-
itive Lyapunov exponent: a nonlinear analysis of the human electroencephalo-
gram during sleep. BioI. Cybern., 69, pp.139-146.
[111] Fell, J., P. Beckmann (1994). Resonance-like phenomena in Lyapunov calcula-
tions from data reconstructed by the time-delay method. Phys. Lett. A, 190,
pp.172-176.
[112] Sato, S., M. Sano, Y. Sawada (1987). Practical methods of measuring the gen-
eralized dimension and largest Lyapunov exponent in high dimensional chaotic
systems. Prog. Theor. Phys., 77, pp.1-5.
[113] Kurths, J., H. Herzel (1987). An attractor in solar time series. Physica D, 25,
pp.165-172.
[114] Diimmig, M., F. Mitschke (1993). Estimation of Lyapunov exponents from time
series: the stochastic case. Phys. Lett. A, 178, pp.385-394.
[115] Rosenstein, M.T., J.J. Collins, C.J. de Luca (1993). A practical method for
calculating largest Lyapunov exponents from small data sets. Physica D, 65,
pp.117.
[116] Kantz, H. (1994). A robust method to estimate the maximal Lyapunov expo-
nent of a time series. Phys. Lett. A, 185, pp.77-87.
[117] Fujisaka, H.,T. Yamada (1993). Stability theory of synchronized motion in
coupled-oscillator systems. Prog. Theor. Phys., 69, pp.32-46.
[118] Singer, W. (1993). Synchronization of cortical activity and its putative role in
information processing and learning. Annu. Rev. Physiol., 55, pp.349-374.
[119] Ashwin, P., J. Buescu, I. Stewart (1994). Bubbling of attractors and synchro-
nisation of chaotic oscillators. Phys. Lett. A, 193,pp.126-139.
[120] Heagy, J.F., T.L. Carroll, L.M. Pecora (1994). Synchronous chaos in coupled
oscillator systems. Phys. Rev. E, 50, pp.1874-1885.
[121J Lai, Y.-C., C. Grebogi (1994). Synchronization of spatiotemporal chaotic sys-

tems by feedback control. Phys. Rev. E, 50,pp.1894-1899.
[122] Collins, J.J.,I. Stewart (1994). A group-theoretic approach to rings of coupled
biological oscillators. Bioi. Cybern., 71, pp.95-103.
[123J Lindner, J.F., B.K Meadows, W.L. Ditto, M.E. Inchiosa, A.R Bulsara (1995).
Array enhanced stochastic resonance and spatiotemporal synchronization.
Phys. Rev. Lett., 75, pp.3-6.
[124J Braiman, Y, W.L. Ditto, K Wiesenfeld, M.L. Spano (1995). Disorder-enhanced
synchronization. Phys. Lett. A 206, pp.54-60j Braiman, Y, J.F. Lindner, W.L.
Ditto (1995). Taming spatiotemporal chaos with disorder. Nature, 378, pp.465-
467.
[125J Pecora, L.M., T.L. Carroll (1990). Synchronization in chaotic systems. Phys.
Rev. Lett., 64, pp.821-824.
[126J Brown, R, N.F. Rulkov, E.R Tracy (1994). Modelling and synchronizing
chaotic systems from experimental data. Phys. Lett. A, 194, pp.71-76.
[127] Kocarev. L., U. Parlitz (1995). General approach for chaotic synchronization
with applications to communication. Phys. Rev. Lett., 74(25), pp.5028-5031.
[128J Parlitz, U., L. Junge, L. Kocarev, (1996). Synchronization based parameter
estimation from time series. Phys. Rev. E, 54, pp.6253-6529.
[129J Parlitz, U., L. Kocarev, T. Stojanovski, H. Preckel (1996). Encoding messages
using chaotic synchronization. Phys. Rev. E, 53(5), pp.4351-4361.
[130J Rulkov, N.F., KM. Sushchik, L.S. Tsimring, H.D.I. Abarbanel, (1995). Gen-
eralized synchronization of chaos in directionally coupled chaotic systems.
Phys. Rev. E, 51, pp.980-994.
[131] Kocarev, L., U. Parlitz (1996). Generalized synchronization, predictability and
equivalence of uni-directionally coupled dynamical systems. Phys. Rev. Lett.,
76(11), pp.1816-1819.
[132J Abarbanel, H.D.I., N.F. Rulkov, M.M. Sushchik (1996). Generalized synchro-
nization of chaos: The auxiliary system approach. Phys. Rev. E, 53(5), pp.4528-
4535.
[133] Parlitz, U., L. Junge, L. Kocarev (1997). Subharmonic entrainment of unstable
period orbits and generalized synchronization. Phys. Rev. Lett., 79(17), pp.3158.
[134J Parlitz, U., L. Kocarev (1998). Synchronization of chaotic systems. in: "Control
of Chaos" Handbook (Ed. H.-G. Schuster), Wiley.
[135J Sirovich, L. (1989). Chaotic dynamics of coherent structures. Physica D, 37,
pp. 126-145.
[136J Rico-Martinez, R., K Krischer, I.G. Kevrekidis, M.C. Kube, J.L. Hud-
son, (1992). Discrete - vs. continuous-time nonlinear signal processing of Cu
electro dissolution data. Chern. Eng. Comm. 118, pp.25-48.
[137J Parlitz, U. & G. Mayer-Kress (1995). Predicting low-dimensional spatiotempo-
ral dynamics using discrete wavelet transforms. Phys. Rev. E, 51(4), pp.R2709-
R2711.

Nonlinear Time-Series Analysis: Ulrich Parlitz

Uploaded by

Copyright:

Available Formats

You might also like

Nonlinear Time-Series Analysis: Ulrich Parlitz

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Nonlinear Time-Series Analysis: Ulrich Parlitz

Uploaded by

Copyright:

Available Formats

8 NONLINEAR TIME-SERIES

Drittes Physikalisches Institut, Universitat Gottingen

Abstract: This tutorial review presents an overview of the achievements and

space reconstruction methods were applied to scalar time series. A mathemat-

8.2 STATE SPACE RECONSTRUCTION

attractor in the reconstruction

mea ured time eries

Figure 8.1 Delay reconstruction of states from scalar time series.

Figure 8.2 Colpitts Oscillator.

In our experimental and numerical implementation of this circuit we use

8.3 INFLUENCE OF THE RECONSTRUCTION PARAMETERS

8.3.1 Choice of the delay time

. ...... . . ....... . . .:.

The situation improves when tl is increased. Then the attractor unfolds

(a) . - - - - - -" (b) __------r,

(c)------~ (d) __------r,

mechanism that is typical for chaotic systems with sensitive dependence on

Xn+l 1- ax~ + bYn (8.3)

8.3.2 Choice of the embedding dimension

8.3.3 Estimating reconstruction parameters

class of algorithms directly considers the geometry of the reconstructed attrac-

8.4 NOISE REDUCTION

8.4.1 Linear Filters

a concentration of states along the diagonal of the reconstructed state space

8.4.2 Nonlinear signal separation

8.5 DETECTING NONLINEARITIES

8.6 MODELLING AND PREDICTION

8.7 FRACTAL DIMENSIONS

8.7.1 Correlation dimension

of the correlation sum

that counts the relative number of neighbouring points closer than r (H is

8.7.2 Information dimension

capacity dimension Do ~ _ logN

and an estimate of the

1000 2000 5000

Figure 8.7 Estimate of the information dimension using Eq . (8.4) .

8.8 LYAPUNOV EXPONENTS

8.8.1 Jacobian-based methods

• using nonlinear approximations of the flow by polynomials [100, 101, 102,

• using projections onto linear subspaces based on local Broomhead-King

8.8.2 Direct methods

8.9 SYNCHRONIZATION OF CHAOTIC DYNAMICS

8.9.1 Estimating model parameters

8.9.2 Generalized synchronization

drives a Lorenz system

by means of a coupling signal u = Xl + X2 + X3. For this coupled system

8.10 SPATIO-TEMPORAL TIME SERIES

8.10.1 Linear decomposition into spatial modes

and performing a singular value decomposition of the resulting N x M matrix

R = U . diag(wl, ... , w m ) . V T . (8.7)

8.10.2 Local models

Local Reconstruction of States. For reasons of simplicity and clear pre-

Nonlinear Prediction. As an application of the local state reconstruction

and the average prediction error E(r)

8.10.3 Numerical Example

justification of the method in future. Another promising generalization of the

[1] Ott, E. (1993). Chaos in Dynamical Systems. Cambridge Cambridge, University

[2] Thompson, J. M. T., H. B. Stewart (1986). Nonlinear Dynamics and Chaos.