System Identification

Control Engineering B.Sc., 3rd year

Technical University of Cluj-Napoca
Romania

Lecturer: Lucian Busoniu

Common input signals

Signal properties

Part VII
Input signals

Characterizing common inputs

Common input signals

Signal properties

Characterizing common inputs

Motivation

Identification methods require inputs that satisfy certain conditions.

Examples:
Transient analysis requires step or impulse inputs.
Correlation analysis preferably works with white-noise input.
PEM and IV require sufficiently informative inputs the
property was left informal in previous material.

Common input signals

Signal properties

Characterizing common inputs

Topic

In this part:
Several new types of input signals are described.
These signals are characterized by their properties relevant for
system identification.

Common input signals

Signal properties

Table of contents

Common input signals

Step, impulse, sum of sines, white noise
Pseudo-random binary sequence

Signal properties

Characterization of common input signals

Characterizing common inputs

Common input signals

Signal properties

Characterizing common inputs

Step input

Left: Unit step:

(
0 k <0
u(k) =
1 k 0
Right: Step of arbitrary magnitude:
(
0
u(k) =
uss

k <0
k 0

Remark: These are discrete-time reformulations of the

continuous-time variants.

Common input signals

Signal properties

Characterizing common inputs

Impulse input

In discrete-time, we cannot generate arbitrary approximations of the

ideal impulse (left), since the signal can only change values at the
sampling instants.
Right: Simplest practical realization:
(
u(k) =

1
Ts

k =0
otherwise

The continuous-time integral of this signal is still 1.

Common input signals

Signal properties

Sum of sines

u(k ) =

m
X

aj sin(j k + j )

j=1

aj : amplitudes of the m component sines

j : frequencies, 0 1 < 2 < . . . < m
j : phases

Characterizing common inputs

Common input signals

Signal properties

White noise

Recall white noise: different steps uncorrelated.

Characterizing common inputs

Common input signals

Signal properties

Characterizing common inputs

White noise (continued)

In the example figure, values were independently drawn from a

zero-mean Gaussian distribution:


1
x2
u(k ) N (0, 2 ) =
exp 2
2
2 2

Common input signals

Signal properties

Table of contents

Common input signals

Step, impulse, sum of sines, white noise
Pseudo-random binary sequence

Signal properties

Characterization of common input signals

Characterizing common inputs

Common input signals

Signal properties

Characterizing common inputs

Pseudo-random binary sequence (PRBS)

A signal that switches between two discrete values, generated with a

specific algorithm.
Interesting because it approximates white noise (explained later).

Common input signals

Signal properties

Characterizing common inputs

PRBS generator

PRBS can be generated with a linear shift feedback register as in the

figure. All signals and coefficients are binary.
At each discrete step k:
State xi transfers to state xi+1 .
State x1 is set to the modulo-two addition of states on the
feedback path (if ai = 1 then xi is added, if ai = 0 then it is not).
Output u(k ) is collected at state xm .

Common input signals

Signal properties

Characterizing common inputs

Modulo-two addition

Formula/truth table of modulo-two addition:

0 if u1 = 0, u2

1 if u = 0, u
1
2
u1 u2 =

1 if u1 = 1, u2

0 if u1 = 1, u2

=0
=1
=0
=1

...also known as XOR (eXclusive OR)

Remark: such a feedback register is easily implemented in hardware.

Common input signals

Signal properties

Characterizing common inputs

Arbitrary-valued PRBS
To obtain a signal u 0 (k ) taking values a, b instead of 0, 1, shift & scale
the original signal u(k ):
u 0 (k ) = a + (b a)u(k )
Example for a = 0.5, b = 0.8:

Common input signals

Signal properties

State space representation

x1 (k + 1) = a1 x1 (k) a2 x2 (k ) am xm (k )
x2 (k + 1) = x1 (k )
..
.
xm (k + 1) = xm1 (k)
u(k ) = xm (k)
x(k) compactly denotes the state vector of m variables

Characterizing common inputs

Common input signals

Signal properties

Characterizing common inputs

Period of PRBS

The PRBS algorithm is deterministic, so the current state x(k )

fully determines the future output
Period (number of steps until sequence repeats) at most 2m
The identically zero state is undesirable, as the future sequence
always remains 0
Maximum practical period is P = 2m 1
A PRBS with period P = 2m 1 is called maximum-length PRBS.
Such PRBS have interesting characteristics, so they are preferred in
practice.

Common input signals

Signal properties

Characterizing common inputs

Maximum-length PRBS
The period is determined by the feedback coefficients ai .
The following coefficients must be 1 to achieve maximum length (all
others 0):
m
3
4
5
6
7
8
9
10

Max period 2m 1
7
15
31
63
127
255
511
1023

Coefficients equal to 1
a1 , a3
a1 , a4
a2 , a5
a1 , a6
a1 , a7
a1 , a2 , a7 , a8
a4 , a9
a3 , a10

Other variants exist, and coefficients for larger m can be found in the
literature.

Common input signals

Signal properties

Table of contents

Common input signals

Signal properties

Characterization of common input signals

Characterizing common inputs

Common input signals

Signal properties

Characterizing common inputs

Mean and covariance

Given a random signal u(k ), its mean and covariance are defined:
= E {u(k )}
ru ( ) = E {[u(k + ) ][u(k ) ]}
Recall:
Mean and variance of random variables
Related covariance function ru ( ) in correlation analysis

Common input signals

Signal properties

Characterizing common inputs

Mean and covariance: deterministic signal

When the signal is deterministic (e.g. PRBS):
N
1 X
= lim
u(k)
N N
k=1

N
1 X
ru ( ) = lim
[u(k + ) ][u(k ) ]
N N
k=1

Note: limN N1
random signal.

PN

k=1

is the same as E {} for a (well-behaved)

Generalization to vector signals u(k ) Rnu : interpret the sums elementwise,

replace [u(k + ) ][u(k ) ] by [u(k + ) ][u(k ) ]> , an nu nu
matrix.

Common input signals

Signal properties

Characterizing common inputs

Correlation analysis
Next: Persistent excitation.
Motivation: We develop an idealized version of correlation analysis.
Finite impulse response (FIR) model:
y (k ) =

M1
X
j=0

h(j)u(k j) + v (k)

Common input signals

Signal properties

Characterizing common inputs

Correlation analysis: Covariance estimation

Assuming u(k ), y (k ) are zero-mean:

N
1 X
u(k + )u(k)
N N

ru ( ) = lim

k=1

N
1 X
y(k + )u(k)
N N

ryu ( ) = lim

k=1

In practice covariances must be estimated from finite datasets, but

here we work with their ideal values.

Common input signals

Signal properties

Characterizing common inputs

Correlation analysis: Identifying the FIR

Taking M equations to find the FIR parameters, we have:

ryu (0)
ru (0)
ru (1)
. . . ru (M 1)
h(0)
ryu (1) ru (1)

ru (0)
. . . ru (M 2)

h(1)

..
..
..

.
.
.
ryu (M 1)
ru (M 1) ru (M 2) . . .
ru (0)
h(M 1)
Denote the matrix in the equation by Ru (M), the covariance matrix.

Common input signals

Signal properties

Characterizing common inputs

Persistent excitation
Definition
A signal u(k ) is persistently exciting (PE) of order n if Ru (n) is
positive definite.
A matrix A Rnn is positive definite if the scalar h> Ah > 0 for any
nonzero vector h Rn . Note that A must be nonsingular.
Examples:


 
1 0
a
is positive definite. Denote h =
, then
0 1
b
>
2
2
h Ah = a + b .


 
1 2
a
is not positive definite. Counterexample: h =
,
2 1
a
h> Ah = 2a2 .

Common input signals

Signal properties

Characterizing common inputs

Role of persistent excitation

Above, an order M of PE ensured that an FIR model of length M
could be identified.
However, PE plays a role in all parametric system identification
methods. Recall the requirements for the guarantees of prediction
error and instrumental variable methods:
Assumptions

2
The input signal u(k ) is sufficiently informative.

These assumptions can be precisely stated in terms of a large

enough order of PE. The required order is proportional to the number
of parameters n that must be estimated.

Common input signals

Signal properties

Characterizing common inputs

Covariance alternatives

When u(k ) is not zero-mean, the two definitions:

N
1 X
u(k + )u(k)
N N

ru ( ) = lim

(7.1)

k =1

N
1 X
ru ( ) = lim
[u(k + ) ][u(k) ]
N N

(7.2)

k =1

can lead to different orders of PE (larger by 1 for the first definition).

We will use the first definition when examining PE in the sequel.

Common input signals

Signal properties

Table of contents

Common input signals

Signal properties

Characterization of common input signals

Characterizing common inputs

Common input signals

Signal properties

Step input

Take the more general, non-unit step:

(
0
k <0
u(k) =
uss k 0

Characterizing common inputs

Common input signals

Signal properties

Characterizing common inputs

Step input: Mean and covariance

Mean and covariance:

N1
1 X
u(k ) = uss
N N

m = lim

k=0

ru ( ) = lim

1
N

N1
X

2
u(k + )u(k) = uss

k=0

Note the signal starts from k = 0, so the summation is modified (unimportant

to the final result).

Common input signals

Signal properties

Characterizing common inputs

Step input: Order of PE

Covariance matrix:

ru (0)
ru (1)

Ru (n) =
..

ru (1)
ru (0)

...
...

ru (n 1) ru (n 2) . . .

2
uss
ru (n 1)
2
uss
ru (n 2)

=
..
.
2
ru (0)
uss

This matrix has rank 1, so a step input is PE of order 1.

2
uss
2
uss

...
...

2
uss

...

2
uss
2
uss

2
uss

Common input signals

Signal properties

Impulse input

Recall discrete-time realization:

(
u(k) =

1
Ts

k =0
otherwise

Mean and covariance:

N1
1 X
u(k ) = 0
N N

m = lim

k=0

1
N N

ru ( ) = lim

N1
X

u(k + )u(k) = 0

k=0

The impulse is not PE of any order.

Characterizing common inputs

Common input signals

Signal properties

Characterizing common inputs

Sum of sines

u(k ) =

m
X

aj sin(j k + j ),

0 1 < 2 < . . . < n

j=1

Mean and covariance:

(
a1 sin(1 )
m=
0

if 1 = 0
otherwise
(
m1
2
X aj2
am
sin2 m
if m =
ru ( ) =
cos(j ) + a2
m
2
2 cos(m ) otherwise
j=1

Common input signals

Signal properties

Characterizing common inputs

Sum of sines (continued)

For the multisine exemplified before, the covariance function is:

A multisine with m components is PE of order n:

if 1 6= 0, m 6=
2m
n = 2m 1 if 1 = 0 or m =

2m 2 if 1 = 0 and m =

Common input signals

Signal properties

Characterizing common inputs

White noise: Mean and covariance

Take a zero-mean white noise signal of variance 2 , e.g. drawn from

a Gaussian:


1
x2
2
u(k ) N (0, ) =
exp 2
2
2 2
Then, by definition:
=0
(
2
ru ( ) =
0

if = 0
otherwise

Common input signals

Signal properties

Characterizing common inputs

White noise: Covariance example

Covariance function of white noise signal exemplified before:

Common input signals

Signal properties

Characterizing common inputs

White noise: Order of PE

Covariance matrix:

ru (0)
ru (1)
..
.

ru (1)
ru (0)

...
...

Ru (n) =

ru (n 1) ru (n 2) . . .

0 ... 0
0 2 . . . 0

= .
= 2 In

..
0

...

ru (n 1)
ru (n 2)

ru (0)

where In = the identity matrix, positive definite.

for any n, Ru (n) positive definite white noise is PE of any order.

Common input signals

Signal properties

Characterizing common inputs

PRBS: Mean

Consider a 0, 1-valued, maximum-length PRBS with m bits:

P = 2m 1, a large number.
Then its state x(k) will contain all possible binary values with m digits
except 0.
Signal u(k) is the last position of x(k), which takes value 1 a number
of 2m1 times, and value 0 a number of 2m1 1 times.
Mean value:
=

P
1X
1
(P + 1)/2
1
1
u(k ) = 2m1 =
= +
2
P
P
P
2 2P
k =1

Common input signals

Signal properties

PRBS: Covariance
Consider a zero-mean PRBS, scaled between a and a:

u 0 (k ) = a + 2au(k)
Then:

1
1
a
= a + 2a( +
)=
0
2 2P
P
(
1 P12 1
if = 0
ru 0 ( ) =
1
1
1
P P 2 P 0 otherwise

Characterizing common inputs

Common input signals

Signal properties

Characterizing common inputs

PRBS: Covariance example

Covariance function of the zero-mean PRBS above:

So, PRBS behaves similarly to white noise (similar covariance

function). This makes it very useful in system identification.

Common input signals

Signal properties

Characterizing common inputs

PRBS: Order of PE

A maximum-length PRBS is PE of exactly order P, the period

(and not larger).