1 Linear Systems, Superposition, and Convolution

In this section we provide a brief introduction to linear systems theory. It is based on the following
definition:
Let L denote a time-invariant, linear system with input and output :





Such systems are additive (the response to the sum of two inputs
responses to each input taken individually):



   
  
 

and

equals the sum of the

and homogeneous (the system can be scaled by the magnitude of the input , where

    
 

is a scalar):

Taken together, they enjoy the Principle of Superposition:

     
 
 

Several idealized input functions are of special importance in analyzing systems, the Dirac delta
function and the unit step function. The unit step is defined as:

!#"

   & "

and the Dirac delta function is its derivative:

undefined

"

) +*-,  /  .
,

if
if
if

%$ ! ;
%' ! ;
 !

(1)

) 

Of course, since the step function is not continuous (notice the value at 0), one has to be careful in
defining exactly what is meant by the above equation. Formally, the
is a distribution defined by the
integral equation:

021

154        4  !  .
,
3
(76 of functions such as:
It can be represented as the limit of a sequence
  & :9<;>= 8 
8
?
with ? 
! , see Fig. 1; During this limiting process we have something that resembles a physical
approximation to the unit impulse, the first test pattern used to evaluate the lateral inhibitory network.
(
@
The result is called the impulse response
 
A  CB
1

2
1.5

0.5
1
0.5
0
0
0.5
20

10

0
t(epislon=1)

10

0.5
20

20

10

0
10
t(epislon=1/2)

20

10

0
10
t(epislon=1/16)

20

2.5
2

1.5

1.5
1

0.5

0.5
0

0
0.5
20

0.5
10

0
10
t(epislon=1/4)

1
20

20

Figure 1:
Similarly, since the step function is the integral of the impulse function, the step response, or the
response to a unit step function

DFEG<HIKJEMLNEG<H<H

DFE)G<H+IKOQP EXWHZYTW[

is the integral of the impulse response function:

RTSVU
In general, of course, we are interested in the response of a system not to these special functions, but
to an arbitrary input. The trick is to approximate this arbitrary input as a sequence of step functions (Fig.
2.12 in Oppenheim and Willsky, p91), and then to use superposition to obtain the overall response. It is
necessary to assume that the input function
is continuous, so that the approximation is meaningful.
Now, suppose the input starts at
, and time is discretized into bins
seconds apart. The first
step in the approximation has height
, a constant, and the additional step at
has height
. Remembering that
, the signal
can thus be approximated by

G]I_^

Efc

e \ `hgi\

EEXc

gkj m` l
E)G<HoI XE ^aHpLNEG<Hq
\ n I \ XE ^aHpLNEG<Hq
\

EG<H

EM^aH \
LNEX^aHI

EG<H

j
\
E H ME ^aH LNEG Hq XE s H
e \ ` Efgrc \ H l EEXc g` H He \ LN)E ` G
wxzS y{}|
\ ` i
g \ g~j `
g

The approximate output is then given by

GbIdc

`
E H LNEG s Hqutvtvt
gc \ H` l g `
r
`

(2)
(3)

 <o NMaF)< } NX#iN<fm~ZzFN#

NMaF)< } NX#iN <fm~Z FN#<
T

(4)
(5)

Now, introduce a limiting process in which the time steps become vanishingly close, so that
, the sum becomes an integral and we have:

 > C ) <o

<
NXF<Q N ) F)i<
p
for a continuous input N< starting at 
N)CRepeating
<)iCthis
approximation
as C#  procedure
yields:
)<o

aN) )i<

N )
N <i<

,
(6)
(7)

and such that

(8)
(9)
(10)

Thus the output can be computed as an integral function of the impulse response. Such an integral is
called a convolution integral, and the above expression is often abbreviated:

)<N)<<

Reference
Alan V. Oppenheim and Alan S. Willsky, Signals and Systems, second edition, Prentice-Hall Inc.,
1997.