ECNG 4302 Fundamentals of

Communications II
Dr. Ahmed Khattab
Signal Space Analysis

Electronics Engineering Department

American University in Cairo

Signal Space Analysis

• Used for analysis of digital signals

• We consider M- ary sources: where the source output

si(t) may be one of M waveforms i= 1,…,M

ECNG 4302 A. Khattab

1
 Geometric Representation of Signals
• We seek to express the signal waveform: s(t) as a linear
combination of a set of orthonormal functions.
• A set of functions  j ( t )  is orthonormal over the
interval 0 , T s  if
Ts

 0
j (t ) l (t ) dt  0 if jl

• Hence, 1 if jl
N
s(t )   s j j (t ) 0  t  Ts
j 1
Ts

where s j   s(t ) j (t )dt

0
3
and j  1,..., N
ECNG 4302 A. Khattab

• Now if we have M signals {si(t)} then the coefficients s ij   N

j 1
are the vector
representation of si(t).

Ts

sij   si (t ) j (t )dt
0
• We may represent si(t) by si= [si1,si2,….siN]T and we can obtain each
form from the other.
1 (t ) 1 ( t )
si1 T

x x 
0
dt si1

Σ
T
si2 x
.
si(t)
si(t) x
. 
0
dt si2
 2 (t ) . .
. 2 (t ).
T

siN x x  0
dt siN

 N (t )  N (t )

•  j ( t ) j  1 are also called a set of basis functions for the set of

N
4
interest  s i ( t ) M
ECNG 4302 i 1 A. Khattab

2
 Example: Fourier Series Analysis of
square wave
• are
orthonormal basis functions.

• So the signal vector for Wo(t) could be represented as

• Wo=[0, 4/π, 4/3π, 4/5π, 4/7π, …]

• In this case, M= …, N= …
5

ECNG 4302 A. Khattab

Linear Independence
• A set of elements (vectors) is said to be linearly independent
(LI), if no element (vector) in the set can be represented as a
linear combination of the other elements (vectors) in the set;
otherwise, we say the set is linearly dependent (LD)

• Examples
y x
x y
(LD) 3) (LD)
1) z
x x
2) y (LI) 4) (LI)
y Orthogonal 6

ECNG 4302 A. Khattab

3
 Examples (Cont’d)
2A
A

5) t t
LD
T T

A A

t
6) LI
2T T 2T t

A A
LI & Orthogonal
7) t
T T 2T t
7

ECNG 4302 A. Khattab

Gram- Schmidt Orthogonalization

• A method for obtaining a basis set j ( t ) Nj 1 from a given signal
set  s i ( t ) iM 1
Ts
1. Start by setting 1 (t )  s1 (t ) / E1 , E1   s12 (t ) dt
0

2. Now we obtain the projection of s2(t) on to 1 (t ), we compute

Ts

s21   ...........................dt
0
Then, the projection is ……………..

ECNG 4302 A. Khattab

4
 3. Define an intermediate function g2(t) as follows
g2(t)
g 2 (t )  s2 (t )  s211 (t )
s2(t)  2 (t )
g 2 (t )
Define 2 (t ) 
Eg2 s 21 1 ( t )
s1(t)
1 (t )
Now, 2 (t )  1 (t ) & has unit energy

4. Continuing in this manner we define

i 1
g i (t )  s i (t )   s ij  j (t )
j 1

g i (t )
 j (t ) 
E gi
We proceed till i = M
• If the set of functions {si(t)} was LI, N=M. Otherwise N < M.
9

ECNG 4302 A. Khattab

Example
• Obtain a basis set for si(t) shown & represent si(t) in terms of the
basis set.

s1(t)
s2(t) s3(t)

A A A

2T
T t T 2T t t
T

• Note: Ts=2T
-A
10

ECNG 4302 A. Khattab

5
 Solution
s1 (t ) s (t )
1 (t )   1
ES1 A2T
Ts

s21   s2 (t )1 (t )dt  0

0

g 2 (t )  s2 (t )  s211 (t )  s2 (t )
g 2 (t ) A 1
 2 (t )    T  t  2T
Eg2 AT 2
T
 0 elsewhere
g3 (t )  s3 (t )  .............. ................
Ts Ts
A
s31   ...............dt  TA T , s32   s3 (t )2 (t )dt ........................
0 T 0
0T T 2T 0T T 2T
g3 (t )  A A A A 0 11
 (t )  0
ECNG 4302 3 A. Khattab

• The basis set is 1(t) & 2(t) & N  2

s1 (t )  s111 (t )  A T 1 (t )
s2 (t )  s222 (t )  A T 2 (t )
s3 (t )  A T 1 (t )  A T 2 (t )

 2 (t )


S1  A T 0 T
A T S2

S  0
2 A T T
S1  1 (t )

S 3  A T -A T  T A T

A T
S3
12

ECNG 4302 A. Khattab

6
 Some Measures in Signal Space
• recall that Si is a representation of si(t) in N-dimensional signal
space with N┴ axes Ø1(t) through ØN(t)

• The set  s i ( t ) i  1 may be represented by M points in signal space.

M

We call this the signal constellation

• Norm of Si:
N
Si  S  Si  T
i s
j 1
2
ij " length" of Si
Energy of Si:

2 N
Ei  S i   sij2
13
j 1
ECNG 4302 A. Khattab

• Proof:
Recall that
2
Ts
 N Ts

Ei   s (t ) dt     sij j (t )  dt
2
i
0 0  j 1 
Ts N N
 s 
0 j 1
ij j (t )   sill (t )dt
l 1

N N Ts

  s s  ................dt
ij il
j 1 l 1 0
N 2
  ..........  S i
j 1

14

ECNG 4302 A. Khattab

7
 • Normalized cross- correlation:
im between si (t ) & sm (t )
Ts

S S
T  s (t )si m (t )dt
im  i m  0
Exercise : Prove equality
Si  Sm Ei Em
 1  im  1
Note : im  Cos(im ) : where  im is the " angle" between S i & S m
 S i  S m  S i  S m Cos(im )
T

Hence, for two orthogonal signals S i & S m , im  0

• The distance dim between si(t) & sm(t):

d im  S i  S m
Ts

  [ si (t )  sm (t )]2 dt
2 15
d im2  S i  S m
ECNG 4302 0 A. Khattab

Example
• Calculate all Ei’s, cross- correlations, & distances for the signal set in
the previous example

  AT
E1  A T
2 2

 A T   A T
2 2
E2
 A T   A T   2 A T
2 2 2
E3

T
A T 0   AA TT 
S1 S 3 
2
  A T  1
 13  
S1  S 3 ( A T )( A 2 T ) A 2T 2 2

• Exercise: Obtain ρ12, ρ23 16

ECNG 4302 A. Khattab

8
 d 12  S 1  S 2  (......... .....) 2
 (......... .......) 2

 A T 0   0  A
2
T 
2
 A 2 ( 2T )
 A 2T

• Exercise: Obtain d23, d13

17

ECNG 4302 A. Khattab

Geometric Representation of Noisy Signals

• Consider the following received signal

x(t) = si(t)+w(t) 0≤ t ≤ Ts
i = 1… M
• Where: w(t) is a stationary zero mean additive white Gaussian noise
process with PSD weight No/2
• The optimum receiver consists of the following "correlator bank"
1(t)

Estimates or
T

x
 dt X1
T
0
decision
x .  dt X Decision ^
xi(t) X2 Device S
. 0

2 (t ) . T

x  dt XN
18
0

N (t )

ECNG 4302 A. Khattab

9
 • Note:
• The decision as to which message was sent is based
on X only
• At the output of the correlator bank
Ts

X j   [s
0
i ( t )  w ( t )] j ( t ) dt  s ij  w j j  1,..., N

Ts

where : w j   w ( t )
0
j ( t ) dt

We can write X  Si  w
N
x (t )  x j 1
j j ( t )  w ( t ) remainder noise term

N
w ( t )  x ( t )  x
j 1
j j (t )
19

• Hence, w’(t) is the part of w(t) outside the span of

 j ( t ) j  1
N

N
Therefore, w(t )  xi (t )  w(t )   ( xik  wk )k (t )
k 1
N
 w(t )   wkk (t )
k 1
• It can be shown that w’(t) is irrelevant to the
decision based on X E[xjw’(t)]=0  j

20

ECNG 4302 A. Khattab

10
 Statistical Properties of Correlator
Output
1) E [ x j | S i was sent ]  E [ s ij  w j ]  .......  E [ w j ]
 s ij
2) Var [ x j | S i ]  E [( x j  E [ x j ]) 2 | S i ]  E [( x j  s ij ) 2 | S i ]
 E [ w 2j ]
Ts Ts

 E [  w (t ) j (t ) dt   w ( ) j ( ) d  ]
0 0
Ts Ts

  
0 0
j (t ) j ( ) E [ w (t )  w ( )] dtd 

21
   j (t ) j ( ) R w (t ,  ) dtd 
ECNG 4302 A. Khattab

where Rw (t , ) is the autocorrelation function

of the r.p w(t)
Rw (t , )  Rw (t   )  w(t) is stationary
N0
 (t   )  w(t) is white with PSD weight N 0 / 2

2
N
Var[ x j ]  0   j (t ) j ( ) (t   )dtd 
2
N N
 0   2j ( ) d  0
2 2
3) Cov[ x j , xk | S i ]  E[( x j  E[ x j ])( xk  E[ xk ]) | S i ]
 E[( x j  sij )( xk  sik )]  E[ w j  wk ]
T
No
2 0
  j (t ) k (t ) dt
22
 0 for j  k
ECNG 4302 A. Khattab
 x j & xk are uncorrelated

11
 • Important Note:
• Since {xj} are Gaussian & uncorrelated, then they are independent.

• Using the above result, we can Prove the earlier result

E[Xj w'(t)] = 0 for all j.

23

ECNG 4302 A. Khattab

Equivalence of Correlator Bank to

Matched Filter Bank
T
y1(t)
1 (T  t ) y1(T)

y2(t) T
 2 (T  t ) y2(T)
x(t)

yN(t) T
 N (T  t ) yN(T)

y j (t )  x (t )   j (T  t )   x( ) (T  t   )d

j

 T
y j (T )   x( ) ( )d   x( ) ( )d  X

j
0
j j

24
• Note: this is true at t=T
ECNG 4302 A. Khattab

12
 The Decision Device
• We now seek a mapping (decision rule) from the observation vector
X to an estimate mˆ (or Sˆ ) of the transmitted symbol mi (or S i ) so
that the decision error probability is minimized.

• The optimum decision rule is given by:

• Set mˆ  m k if Pr (mk was sent | x) ≥ Pr(ml was sent |x)
for all l ≠ k

• This is known as the Maximum A Posteriori (MAP) decision rule.

mˆ  arg max Pr[ mk sent | X ]

mk

marginal PDF marginal PDF

Recall that p ( m k | x ) f ( x )  f ( x , m k )  f ( x | m k )  p r ( m k ) 25

ECNG 4302 Conditional PDF Joint PDF Conditional PDF A. Khattab

Joint PDF Conditional PDF marginal PDF

f ( x, m k ) f ( x | mk )  pr (mk )
p (mk | x)  
f ( x) f (x)
 The MAP rule is equivalent to
  f ( x | mk )  pr (mk ) 
mˆ  arg  max m k    Likelihood Fn
  f (x) 

 arg max m k  f ( x | m k )  p r ( m k )  
for equiprobab le symbol transmiss ion, P r ( m k )  1 / M

 mˆ  arg max m k  f ( x | m k )  
• This form is known as Maximum Likelihood (ML)
decision rule.
• It is more convenient to work with the log() of the
likelihood function
• Recall that {xj} are jointly Gaussian & we proved their
independence. 26

ECNG 4302 A. Khattab

13
 f ( X | m k )  f ( x1 | m k )  f ( x 2 | m k )... f ( x N | m k )
N N
1
  f ( x j | mk )  
( x  S )2 / N 0
e j kj
j 1 j 1 N 0
N
1

N0  ( x j  S kj ) 2
 (N 0 ) N /2
e j 1

1 N
 ln  f ( X | m k )   ln(N 0 )  N / 2   ( x j  S kj )
2

N 0 j 1
  1 N 2
 GaussianPDF
 mˆ  arg  max mk   ( x j  s kj )    ( x  x )2 
1
  N0 j  1
 f x ( x)  exp 
2 2
 2 x2 
 
x
 N
 arg  min mk   ( x j  s kj ) 2   x  skj
 
j
j 1

  X  S 
 xj2  No / 2
2
 arg min mk k for independent RP's x, y,

 arg min mk X S k  the joint PDFis given by
f x, y ( x, y)  f x ( x) f y ( y)

• ie. Set mˆ  mk if X  S k is smallest for k.

27
• This is known as the minimum distance criterion.
ECNG 4302 A. Khattab

Decision Regions and Decision

Boundaries
• Based on the minimum distance rule, the signal space may be
divided into M decision regions denoted by: Zi ,i=1,…,M

• The decision rule becomes:


If X  Z i , set m  m i

• Example (4-psk = QPSK)

M=4. Therefore, we have 4 decision regions.

28

ECNG 4302 A. Khattab

14
 Error Probability Computation
• Assume mi was transmitted
Pe|mi  Pr ( X  Zi | mi )
M From theorem.................
Pe   Pe|mi  Pr (mi )  
i 1  P(A)   P( A | Bi )P( Bi ) 
 i 
1 M
  f ( X | mi )dx
M i 1 Zc
i

• Sometimes, it is easier to calculate the probability of a correct

decision Pc
M
1
Pc 
M
  f ( X | m )dx
i 1 Z
i
i 29
Then, Pe  1  Pc
ECNG 4302 A. Khattab

• Notes:
• The probability of error depends only on the
relative Euclidean distances between signal points
in the constellation
• Hence, rotation and translation do not affect Pe

• Notes:
• The average symbol energy (Eav) in a constellation is
minimum when the origin is at the center of gravity
of the constellation. 30

ECNG 4302 A. Khattab

15
 Example: Probability of Error Calculation
• Assume PCM system with NRZ Line Coding.
• We may represent transmitted signal by:

• Assume Tb is the bit duration, and assume perfect bit synchronization.

• Given the noisy input, the RX is required to make a decision in each
signaling interval whether 1 or 0 is transmitted.
• Two possible types of error could exist: Decide 1 while 0 is actually
transmitted (error of 1’st kind – False Alarm), and decide 0 while 1 is
transmitted (error of 2’nd kind – Miss detection)
• We need to calculate the average probability of detection error.

31

ECNG 4302 A. Khattab

32

ECNG 4302 A. Khattab

16
33

34

17
35

18