ELEN E4810: Digital Signal Processing

Topic 2: Time
1. domain systems
Discrete-time
2. Convolution
3. Linear Constant-Coefficient Difference
Equations (LCCDEs)
4. Correlation

1
 1. Discrete-time
systems
 A system converts input to output:

x[n] DT System y[n] {y[n]} = f ({x[n]}) A

n

 E.g. Moving Average (MA):

1/M
x[n]
z-1 y[n] 1 M 1 x[n 
x[n-1]
z-1
1/M
+ y[n]
 M  k0
k]
1/M
x[n-2] (M = 3)
2
 Moving Average
(MA) x[n] A

1/M
-1 1 2 3 4 5 6 7 8 n
9
x[n] x[n-1]
z-1
1/M y[n]
x[n-1] + -1 1 2 3 4 5 6 7 8 n
z-1 9
1/M x[n-2]
x[n-2]
-1 1 2 3 4 5 6 7 8 9
n

1 M y[n]
y[n]  1 x[n 
M k0
k] -1 1 2 3 4 5 6 7 8 9
n
3
MA Smoother
 MA smoothes out rapid variations
(e.g. “12 month moving
 average”)
e.g. signal noise
x[n] = s[n] + d[n] y[n]  1 k0 x[n  k]
5 5-pt
4 moving
 average

4
Accumulator
 Output accumulates all past inputs:
n

y[n]  
 x[n] + y[n]
x[]  z-1
n1

 x[] 
  y[n-1]

 y[n 1] 
x[n]
x[n]
5
Accumulator
x[n] A

-1 1 2 3 4 5 6 7 8 n
9
y[n-1]
x[n] + y[n]
z-1
-1 1 2 3 4 5 6 7 8 9
n
y[n-1]
y[n]

-1 1 2 3 4 5 6 7 8 9
n

6
M
Classes of DT systems
 Linear systems obey superposition:
x[n] DT system
y[n]
 if input x1[n] → output y1[n], x2 → y2 ...

given a linear combination
of inputs:
 then output
for all ,  x1 x2
i.e. same linear combination of outputs
7
Linearity: Example 1
n
 Accumulator:

y[n] 
x[n]    x1[n]    
x [n] n x[]
2
y[n] 
x1[] 
x2[]


   y1[n]    Linear
 x1[]
y2[n]
8
Linearity Example 2:
 “Teager Energy operator”:
y[n]  x [n]  x[n 1]  x[n
2

1]
2
  x
x[n]y[n] 1[n]   xx2[n]
x1[n] 2[n]

x1[n 1]  x2[n 1]

x1[n 1]  x2[n
   y1[n] 1]
   Nonlinear
y2[n] 9
Linearity Example 3:
n
 ‘Offset’ accumulator:
n 
y[n]  C 
 y1[n]  C   x1[]
x[] 


n

but y[n]  C  x1[]  x2[]


 y1[n]  Nonlinear
y2[n] .. unless C = 0

10
Property: Shift (time) invariance
 Time-shift of input
causes same shift in output
 i.e. if x1[n] → y1[n]
then x[n]  x1[n  n0 ]
 y[n]  y1[n  n0 ]

i.e. process doesn’t depend on absolute
value of n

11
Shift-invariance counterexample
 Upsampler: x[n] L y[n]
y[ n]  0 
x[ n / L] n  0 ,  L , 2 L
otherwise

y [ n]  x [ n / L]  ( n  r.L)
1 1
x[ n]  x [ n  n ]
1 o
y[ n]  x [ n / L ]  x [ n / L  n ]
1 o
Not shift invariant

 n  L .n
o
x   y [ n  L.n ]  y [ n  n ]
1

 L 1 o 1 o

12
Another counterexample
y[n]  n  scaling by time index
x[n]
 Hence
y1[n  n0 ]  n  n0   x1[n 
 If x[n]  x1[n  ≠
n0 ]
then
n0 ] y[n]  n  x1[n 
n0 ]
 Not shift invariant
- parameters depend on n

13
Linear Shift Invariant (LSI)
 Systems which are both linear and
shift invariant are easily manipulated
mathematically
 This is still a wide and useful class of
systems
 If discrete index corresponds to time,
called Linear Time Invariant (LTI)

14
Causality
 If output depends only on past and
current inputs (not future), system is
called causal
 Formally, if x1[n] → y1[n] & x2[n] →
yCausal
2[n] x [n]  x [n] n  N
1 2

 y1[n]  y2[n] n  N

15
Causality example
1 M
 Moving average: y[n]  1 x[n 
M k0
k]
y[n] depends on x[n-k], k ≥ 0 → causal
 
‘Centered’ moving average
ycn  yn  M  1 / 2
1

M 

x[n] 2
( M 1)/
x[n  k]  x[n  
k 1 k]
 .. looks forward in time → noncausal
 .. Can make causal by delaying
16
 Exercises :
Test the system given by the following Eqs. for
1- linearity 2- causality 3- time invariance
2
 n  2.5 
 y[ n]    x[ n]
 n  1.5 
 y[ n]  x[  n]
 y[ n]  5 x[ n]
 y[ n]  10 n x[ n]
 y[ n]  x[  n  1]
 y[ n]  x[  n  1]
 y[ n]  x[ n  1], x[ n  1]
 y[ n]  x[  n 2 ], x[ n]2
17
Impulse response (IR)
δ[n] 1
 Impulse -3 -2 -1 1 2 3 4 5 6 n
7

(unit sample sequence)

 Given a system: x[n] DT system y[n]

if x[n] = δ[n] then y[n] ∆ h[n]

“impulse response”
 LSI system completely specified by h[n]

18
Impulse response
example
 Simple system: x[n]
α
z-1
α2 y[n]
+
z-1
α3

x[n] 1
-3 -2 -1 1 2 3 4 5 6 n x[n]  δ[n] impulse
7

α
y[n] α α
-3 -2 -1 1 2 3 4 5 6 n y[n] = h[n] impulse response
7

19
2. Convolution
 Impulse response: δ[n LSI h[n]
]
 Shift invariance: δ[n-n0] LSI h[n-n0]

 + Linearity: α· δ[n-k]+ α·h[n-k]

LSI
β· δ[n-l] + β·h[n-l]

Can express any sequence with δ s:



x[n] = x[0] δ[n] + x[1] δ[n-1] + x[2] δ[n-

2]..
20
Convolution

sum
 Hence, since x[n]  

x[k]

[n  k]

k
For LSI, y[n]   x[k]h[n Convolution
sum
 k]
written as k
y[n]  x[n] *
 Summation is symmetric in x and h
h[n]
i.e. l = n – k → 

x[n] * h[n]   x[n  l]h[l]  h[n] *

x[n] 21
Convolution properties
 LSI System output y[n] = input x[n]
convolved with impulse response h[n]
→ h[n] completely describes system

Commutative: x[n] * h[n] = h[n] * x[n]

Associative:
(x[n] * h[n]) * y[n] = x[n] * (h[n] * y[n])

Distributive:
h[n] * (x[n] + y[n]) = h[n] * x[n] + h[n] * y[n]
22
 Interpreting convolution
 Passing a signal through a (LSI) system
is equivalent to convolving it with the
system’s impulse response

x[n] h[n] y[n] = x[n]∗

h[n] 
y[n]   x[k]h[n  k] 
h[k]x[n  k]
x[n]={0
k 3 1 2 -1} k h[n] = {3 2 1}

→
→

-3 -2 -1 1 2 5 6 n -3 -2 -1 1 2 3 4 5 6 n
3 7 7

23
Convolution

interpretation
1
y[n]   x [k]

x[k]h[nk
 k] call h[-n] = g[n] -
3 -2 -1 1 2 3 5 6 7 k
 Time-reverse h, h[0-k]
=g[ k]
shift by n, take inner
product against - 3 -2 -1 1 2 3 4 5 6 7
k
h[1-k]
fixed x =g[k-1]

9 9
11 - 3 -2 -1 1 2 3 4 5 6 7
k
y[n] h[2-k]
2
0 -1
-3 -2 -1 1 2 5 7 n
3
-3 -2 -1 1 2 3 4 5 6
7
k

24
Convolution

interpretation
2y[n]  x[n]
h[k]x[n
k
 k] -3 -2 -1 1 3 6 7 n

h[k]
2 5

1
-3 -2 -
x[n-1]
 Shifted x’s
weighted by -3 -2 -1 1 2 3 4 6 7
n

7
1 2 3 4 5 6
points in h x[n-2]
 Conversely, -3 -2 -1 1 2 3 4
5
7 n
weighted, 9 9
11
k

delayed y[n]
2
0 -1
versions of h ... -3 -2 -1 1 2
3
5 7 n
25
 Matrix interpretation

y[ n]   x[n  k ].h[k ]
k  

 y[0]  x[0] x[1] x[ 2] h[0]

    
 y[1]    x[1] x[0] x[1]. h[1] 
 y[ 2]  x[ 2] x[1] x[0]  h[ 2]
    
.. .. ..
. Diagonals
. in X matrix are
. equal

26
 Convolution
notes
 Total nonzero length of convolving N

and M point sequences is N+M-1

 Adding the indices of the terms within
the summation gives n :
y[n]  k
  n  k 
h[k]x[n
k
 k]
n in
i.e. summation indices move
opposite senses

27
Convolution in MATLAB
 The M-file conv implements the
convolution sum of two finite-length
sequences
 If a  3 1 2 -1]
[0 2 1]
thenbconv(a,b)
 yields
[3
[0 9 9 11 2 0 -1]

28 M
Connected
systems
 Cascade connection:

Impulse response h[n] of the cascade of

two systems with impulse responses
h1[n] and h2[n] is h[n]  h1[n]

By commutativity, *

29
Inverse
systems
 δ[n] is identity for

convolution
 Consider
i.e. x[n] * δ[n]  x[n]
x[n] y[n] z[n]

z[n]  h2[n] * y[n]  h2[n] * h1[n] * x[n]

 x[n] h2[n] * h1[n] 

if
 h2[n] is the inverse
[n] system of h1[n]
30
Inverse
systems
 Use inverse system to recover input x[n]

from output y[n] (e.g. to undo effects of

transmission channel)
 Only sometimes possible - e.g. cannot
‘invert’ h1[n] = 0
 In general, attempt to solve
h2[n] * h1[n]  [n]

31
Inverse system
example
 Accumulator:

Impulse response h1[n] = μ[n]

 ‘Backwards difference’ -3 -2 -1 1 2 3 4 5 6 n
7

.. has desired property:

[n]  [n 1]  [n] -3 -2 -1 1 2 3 4 5 6
7
n

 Thus, ‘backwards difference’ is inverse

system of accumulator.

32
Parallel connection

 Impulse response of two parallel

systems added together is:

33
3. Linear Constant-Coefficient
DifferenceEquation (LCCDE)
 General spec. of DT, LSI, finite-dim sys:
N M
 defined by {dk},{pk}
 d y[n  k]   pkx[n 
k0 k  order = max(N,M)
k]
k0
 Rearrange for y[n] in causal form:
N M
dk
y[n]   y[n  k]  k x[n 
p
 k1 d 0
 k0 d 0
k]
 WLOG, always have d0 = 1

34
 Solving
LCCDEs
 “Total solution”

y[n]  yc[n] 
y p [n]
Complementary Solution Particular Solution
N
for given forcing function
satisfies x[n]
 dk y[n  k] 
0
k0

35
Complementary Solution
 General form of unforced oscillation
i.e. system’s ‘natural modes’
n
 Assume yc has form yc[n]  
N
nk

k0 d k  
0
N
Characteristic polynomial
  dk  k
 of system - depends only on {dk}
0
k0 36
Complementary Solution
N

N
factors into roots λi , i.e.
 dk  
k
(  1)(   2 )... 
0 0
 Each/any
k0 λi satisfies eqn.

Thus, complementary
nn solution:
n
yc[n]  11  2 2 
3 3  ...
Any linear combination will
work
→ αis are free to match initial 37
Complementary Solution
 Repeated roots in chr. poly:
(  1 ) (  2 )... 
L

 yc[n]  11 0
n n
 3 n  1
2 n

L1
2 n1 n
...   L n 1
 ... n
 Complex λis → sinusoidal yc[n]   i  i

38
Particular Solution
 Recall: Total solution y[n]  yc[n] y p [n]
  input
Particular solution reflects
 ‘Modes’ usually decay away for large n
leaving just yp[n]
 Assume ‘form’ of x[n], scaled by β:
e.g. x[n] constant → yp[n] = β
x[n] = λ0 → yp[n] = β · λ0 (λ0 ∉ λi)
n n
or = β nL λ0n
(λ0 ∈ λi)
39
LCCDE
example
y[n]  y[n 1]  6y[n  2] 
x[n] y[n]
x[n] +

 Need input: x[n] = 8μ[n]

 Need initial conditions:
y[-1] = 1, y[-2] = -1

40
LCCDE
example
 Complementary solution:

y[n]  y[n 1]  6y[n  2]  y[n] 

0; n
 n 2 2    6  0
   3  2  0 →roots λ1 = -3, λ2 = 2
n n
 yc [n]   1 3   2 2

α1, α2 are unknown at this point

41
LCCDE
example
 Particular solution:

 Input x[n] is constant = 8μ[n]

assume yp[n] = β, substitute in:
y[n]  y[n 1]  6y[n  2]  x[n] (‘large’
     6  8[n] n)
 4  8    2

42
LCCDE
example

Total solution y[n]  yc[n]  y p [n]
n n
  1 3


 DE
initial conditions into 2 2at
 n = 0, 1, ...
y[n]  y[n 1]  6y[n  2]  x[n]
from ICs

 n = 0 y[0]  y[1]  6y[2] 
Solve forx[0]unknown αis by
substituting
      1  6  8
1 2

 1   2  3 43
LCCDE
example
 n = 1 y[1]  y[0]  6y[1] 
x[1]
 1(3)   2 (2)    1   2    6  8
 21  32  18
 solve: α1 = -1.8, α2 = 4.8

Hence, systemn output:n
y[n]  1.8(3)  4.8(2)  2 n 
0
Don’t find α s by solving with ICs at
 i
(ICs may not reflect natural modes;
n = -1,-2 Mitra3 ex 2.37-8 (4.22-3) is wrong)
44 M
LCCDE solving
summary
 Difference Equation (DE):
Ay[n] + By[n-1] + ... = Cx[n] + Dx[n-1] + ...
Initial Conditions (ICs): y[-1] = ...
 DE RHS = 0 with y[n]=λn → roots {λi}
gives complementary soln yc [n]    i  i n
 Particular soln: yp[n] ~ x[n]
solve for βλ0n “at large n”
 αis by substituting DE at n = 0, 1, ...
ICs for y[-1], y[-2]; yt=yc+yp for y[0], y[1]
45
LCCDEs: zero input/zero
state
 Alternative approach to solving

LCCDEs is to solve two subproblems:

 yzi[n], response with zero input (just ICs)
 yzs[n], response with zero state (just x[n])
 Because of linearity, y[n] = yzi[n]+yzs[n]
 Both subproblems are ‘fully realized’
 But, have to solve for αis twice
(then sum them)
46
Example : Determine the total solution for n  0 of the difference equation
y[n]  0.1y[n  1]  0.06y[n  2]  2n [n], with the initial condition y[1]  1 and y[2] 
0. Answer: y[n]  0.1y[n  1]  0.06 y[n  2]  2n [ n] with y[–1] = 1 and y[–2] = 0. The
complementary solution y c [n] is obtained by solving y c [n] 0.1yc[n  1]  0.06y c [n  2] 
0.
To this end we set y c [n]  n , which yields

n  0.1n1 – 0.06 n2  n 2(2  0.1 – 0.06)  0 whose solution gives 1  –0.3 and
 2  0.2 . Thus, the complementary solution is of the form y c [n]  1(0.3)n   2 (0.2) n .
For the particular solution we choose y p [n]  (2)n. ubstituting this solution in the difference
n n n
The totalrepresentation
equation solution is therefore
of the system get  y2cn[n]
given bywey[n]  yp [n]n1
 (0.1)2  (0.06)2n2
– 1(0.3) 2 n [n].
 2(0.2)  207
For 2 .
200
1 1
From
n = 0the
weabove
get y[1]  1
(0.1)2 1(0.3)  2
–  (0.06)2 2 (0.2
 1)or   207 21/ 207
 200  1 and
 0.9662 .
2
y[2]  1 (0.3)   2 (0.2 )  2 200 2
2  0 or equivalently, –10 1  52  107 and200
207 3
100 1  50
9  25 2  207 – whose solution yields  1  –0.1017 and  207
2  0.0356. Hence, the
total

solution is given by y[n]  0.1017(0.3) n  0.0356(0.2)n  0.9662(2)n,

for n  0.

47
Exercises:: Determine the total solution for n  0 of the difference
equation shown below?
 y[n]  0.2 y[n  1]  x[n]
n
 with" IC" , y[0]  0,and " input " x[n]  9(0.7) .
 y[n]  0.3 y[n  1]  x[n]
n
 with" IC" , y[1]  0,and " input " x[n]  (0.6) .

48
Impulse response of
LCCDEs
 Impulse response: δ[n] LCCDE h[n]

i.e. solve with x[n] = δ[n] → y[n] = h[n]

(zero ICs)

 With x[n] = δ[n], ‘form’ of yp[n] = βδ[n]

→ solve y[n] for n = 0,1, 2... to find αis

49
LCCDE IR
example

e.g. y[n]  y[n 1]  6y[n  2]  x[n]
 (from before); x[n] = δ[n]; y[n] = 0 for n<0
n
yc [n]   1 3
n
 2 yp[n] = βδ[n]
2  1
 n = 0: y[0]  y[1]  6y[2]  x[0]

⇒ α1 + α 2 + β = 1
 n = 1: α1(–3) + α2(2) + 1 = 0
⇒ α1 = 0.6, α2 = 0.4, β = 0
n = 2: α1(9) + α2(4) – 1 – 6 = 0 n ≥ 0
 thus Infinite length
h[n]  0.63n  M
50
System property: Stability
 Certain systems can be unstable e.g.

x[n] + y[n]
...
z-1
2 y[n]
-1 1 2 3 n
4
Output grows without limit in some
conditions

51
Stability
 Several definitions for stability; we use
Bounded-input, bounded-output
(BIBO) stable
For every bounded input x[n]  Bx n


output is also subject to a finite bound,

y[n]  By n

52
Stability example
 MA 1
y[n]  M  k0 x[n  k]
M 1

filter:
1
y[n]  M  k0 x[n  k]
M 1

1
 M
M 1
k0 x[n  k]
1
 M  Bx  y → BIBO Stable
B M

53
Stability &
LCCDEs
 LCCDE output is of form:

n n
n
y[n]  11   2  2  ...  0 
...
 αs and βs depend on input & ICs,
but to be bounded for any input
we need |λ  

54
 4. Correlation
 Correlation ~ identifies similarity
between sequences: Cross
correlation
“lag” of x against y

rxy [l ]   x[n]. y[n  l ]
n  

 Note : ryx [l ]   y[n].x[n  l ]
n   call , m  n  l

  y[m  l ]x[m]  r
m  
xy [ l ]

55 M
 Correlation and convolution

 Correlatio n : rxy [n]   x[k ]. y[k  n]
k  

 Convolution : x[n]  y[n]   x[k ]. y[n  k ]
k  

 Hence : rxy [n]  x[n]  y[n]

Correlation may be calculated by

convolving with time-reversed sequence

56
Autocorrelation
 Autocorrelation (AC) is correlation of
signal with itself:


rxx[]   x[n]x[n  ] 
rxx[]

Note: r [0]
xx   x [n] 
n 2
x
Energy of
sequence x[n]
n 

57
Correlation
maxima
 Note: r []  r xx[0]
rxx[

r]xx[0] 1
xx


rxy[
    1
Similarly: rxy[]  x y
rxx][0]ryy[0]

 From geometry,  xi 2
angle
between
x and y
xy   i xiyi  x y cos
 when x//y, cosθ = 1, else cosθ < 1
58
AC of a periodic
sequence

Sequence of period N: x˜[n]  x˜[n  N ]
 Calculate AC over a finite window:
M
1
rx˜x˜[]

2M 1 n  M
x˜[n]x˜[n 
N
1
]
1 if M >> N
 Nn0 x˜[n]x˜[n 

]

59
 AC of a periodic
sequenceN
1 1 2 Average energy per
rx˜ [0] x˜ [n] 
x˜ N  P
x
˜
sample or Power of x
n0

N
1
rx˜x˜ [  1

N]
 x˜[n]x˜[n    N ] 
N n0
rx˜x˜ []
 i.e AC of periodic sequence is
periodic
60
What correlations look like
 AC of any x[n] rxx[]


 AC of periodic rx˜x˜[]


 Cross correlation rxy[]

61 
What correlation looks
like

62
Correlation in
action
 Close mic vs.

video
camera mic

 Short-time
cross-correlation

63
 Problems:

a) Evaluate the autocorrelation sequence of each of the

sequences in the following :

The sequences above?

c) Evaluate the convolution of x[n]*y[n] and y[n]*w[n]