Digital Communications I:

Modulation and Coding Course

Term 3 - 2008
Catharina Logothetis
Lecture 9
 Last time we talked about:
Evaluating the average probability of
symbol error for different bandpass
modulation schemes

Comparing different modulation schemes

based on their error performances.

Lecture 9 2
 Today, we are going to talk about:
Channel coding

Linear block codes

The error detection and correction capability
Encoding and decoding
Hamming codes
Cyclic codes

Lecture 9 3
 Block diagram of a DCS

Source Channel Pulse Bandpass

Format
encode encode modulate modulate

Digital modulation

Channel
Digital demodulation

Source Channel Demod.

Format Detect
decode decode Sample

Lecture 9 4
What is channel coding?

Channel coding:
Transforming signals to improve
communications performance by
increasing the robustness against
channel impairments (noise,
interference, fading, ...)
Waveform coding: Transforming
waveforms to better waveforms
Structured sequences: Transforming
data sequences into better sequences,
Lecture 9 5
 Error control techniques
Automatic Repeat reQuest (ARQ)
Full-duplex connection, error detection codes
The receiver sends feedback to the transmitter,
saying that if any error is detected in the received
packet or not (Not-Acknowledgement (NACK) and
Acknowledgement (ACK), respectively).
The transmitter retransmits the previously sent
packet if it receives NACK.
Forward Error Correction (FEC)
Simplex connection, error correction codes
The receiver tries to correct some errors
Hybrid ARQ (ARQ+FEC)
Full-duplex, error detection and correction codes
Lecture 9 6
 Why using error correction coding?
Error performance vs. bandwidth
Power vs. bandwidth
P
Data rate vs. bandwidth B

Capacity vs. bandwidth

Coded

A
Coding gain: F

For a given bit-error probability, C B

the reduction in the Eb/N0 that can be
realized through the use of code: D
E

E
E Uncoded


G
b


b
[dB]
[dB
[dB


N
0
u
N
0
c E
/
bN(dB
0

Lecture 9 7
 Channel models
Discrete memory-less channels
Discrete input, discrete output
Binary Symmetric channels
Binary input, binary output
Gaussian channels
Discrete input, continuous output

Lecture 9 8
 Linear block codes

Let us review some basic definitions first

that are useful in understanding Linear
block codes.

Lecture 9 9
 Some definitions

Binary field :
The set {0,1}, under modulo 2 binary
addition and multiplication forms a field.
Addition Multiplication
0 0 0 0 0 0
0 1 1 0 1 0
1 0 1 1 0 0
1 1 0 1 1 1
Binary field is also called Galois field, GF(2).

Lecture 9 10
 Some definitions
Fields :
Let F be a set of objects on which two
operations + and . are defined.
F is said to be a field if and only if
1. F forms a commutative group under + operation.
The additive identity element is labeled 0.
a,b F a b b aF
1. F-{0} forms a commutative group under .
Operation. The multiplicative identity element is
labeled 1.
a,b F ab b a F
1. The operations + and . are distributive:
a(b c) (
ab) (a c)
Lecture 9 11
 Some definitions

Vector space:
Let V be a set of vectors and F a fields of
elements called scalars. V forms a vector space
over F if:
1. Commutative: u,
v V u v v u F
2. a F
, v V av u V
3. Distributive:
(a b)v a

v bv an a(u v ) a u
av
4. Associative: a,
b F
, v V (a b)v
a
(b
v)
5. v V,1 v v

Lecture 9 12
 Some definitions
Examples of vector spaces
The set of binary n-tuples, denoted by Vn

V
{(
40000
),
(
0001
),
(0010
),
(
001
),
(010
),
(
01
),
(01
),
(
1000
),
(
1001
),
(
1010
),
(
1011
),
(
110
),
(
11
),
(
11
)}
Vector subspace:
A subset S of the vector space Vn is called a
subspace if:
The all-zero vector is in S.
The sum of any two vectors in S is also in S.

Example:
{(
0000
),
(
0101
),
(
1010
),
(
111
)}
is
a
sub
of
V
.
4

Lecture 9 13
 Some definitions
Spanning set:
A collection of vectors G v ,
1v2 , ,v n , is said to
be a spanning set for V or to span V if
linear combinations of the vectors in G include all
vectors in the vector space V,
Example:


(
1
)
(
0
)
(
1
)
(
0
)
(
1
)
s
V
.
4

Bases:
The spanning set of V that has minimal cardinality is
called the basis for V.
Cardinality of a set is the number of objects in the set.
Example:


(
1
)
(
0
)
(
0
)
(
0
)
i
a
b
f
V
.
4

Lecture 9 14
 Linear block codes

Linear block code (n,k)

A set C Vn with cardinality 2 is called a
k

linear block code if, and only if, it is a
subspace of the vector space Vn .


V
C
kV
n

Members of C are called code-words.

The all-zero codeword is a codeword.

Any linear combination of code-words is a

codeword.

Lecture 9 15
 Linear block codes contd

mapping Vn
Vk
C

Bases of C

Lecture 9 16
 Linear block codes contd

The information bit stream is chopped into blocks of k bits.

Each block is encoded to a larger block of n bits.
The coded bits are modulated and sent over the channel.
The reverse procedure is done at the receiver.

Channel
Data block Codeword
encoder
k bits n bits

n-k Redundant
bits
k
Rc Code
rate
n

Lecture 9 17
 Linear block codes contd
The Hamming weight of the vector U,
denoted by w(U), is the number of non-zero
elements in U.
The Hamming distance between two vectors
U and V, is the number of elements in which
they differ.
d
(U,V
) w (
U V )

The minimum distance of a block code is

dd
(
U
,
U
min
)
min
ijmi
w
(
U
)
i

i
j i

Lecture 9 18
 Linear block codes contd

Error detection capability is given by

ed
min1

Error correcting-capability t of a code is

defined as the maximum number of
guaranteed correctable errors per codeword,
that is
d
1

t
min

2

Lecture 9 19
 Linear block codes contd

For memory less channels, the probability

that the decoder commits an erroneous
decoding is P j
n n
p
M(1
p

n
)
j

j
t
1 j
p is the transition probability or bit error probability
over channel.
The decoded bit error probability is
nn
1
B


P
j

p

j
(
1n
j
p
)
n

j
t
1
j

Lecture 9 20


2
l
M
E
2
l
M
E
R
2
2

2
c
2
b
c
p
Q
s
Q
s

l
M
N
M
l
M
N
M

2
0
2
0

Tx. bits
1

Linear block codes contd
Discrete, memoryless, symmetric channel model

p
1-p

1-p
1

0
Rx. bits

Note that for coded systems, the coded bits are

modulated and transmitted over the channel. For
example, for M-PSK modulation on AWGN channels
(M>2):

where Ec is energy per coded bit, given by E

Lecture 9
c R
cEb

21
 Linear block codes contd

mapping Vn
Vk
C

Bases of C
A matrix G is constructed by taking as its
rows the vectors of the basis, {
V1,V2
, ,V k.
}

v v v
1
11 12 1
n
V


G
v
v
21 22 v
2
n



Vk



v1 v
k 2
k v
kn

Lecture 9 22
 Linear block codes contd

Encoding in (n,k) block code

UmG V1
V
(u,u ,,u )(m ,m , ,m ) 2
1 2 n 1 2 k


Vk

(u
1,u2,,un)m1V 1 2V
m 2 m2V
k

The rows of G are linearly independent.

Lecture 9 23
 Linear block codes contd
Example: Block code (6,3)
Message vector Codeword

0 0 0 0 0 0 0 0 0
V1 110100 1 0 0 1 1 0 1 0 0
G V 011010
2 0 1 0 0 1 1 0 1 0

V3
101001
1 1 0 1 0 1 1 1 0
0 0 1 1 0 1 0 0 1
1 0 1 0 1 1 1 0 1
0 1 1 1 1 0 0 1 1
1 1 1 0 0 0 1 1 1

Lecture 9 24
 Linear block codes contd

Systematic block code (n,k)

For a systematic code, the first (or last) k
elements in the codeword are information bits.
G[P Ik]
Ik kk identity
matrix
Pk k(nk) matrix

U
(
u
,
u
,.
u
)
(
p
,
p
,p
,
m
,
m
,
m
)




1
2
n
1
2



n
k
1
2






k
p
b
m
b

Lecture 9 25
 Linear block codes contd

For any linear code we can find a

matrix H ( n k )n , such that its rows are
orthogonal to the rows of G :
T
GH0
H is called the parity check matrix and
its rows are linearly independent.
For systematic linear block codes:
H [Ink P T
]

Lecture 9 26
 Linear block codes contd

Data source Format

m Channel U Modulation
encoding
channel
Channel Demodulation
Data sink Format
decoding r Detection
m

r Ue

r
(
r
,
r
1
2,....,
r)
rece
n cod
or

e
(
e
,
e
1
2,....,
e)
erro
npatt
or v
Syndrome testing:
S is the syndrome of r, corresponding to the error
pattern e. S r T
e T
HH

Lecture 9 27
 Linear block codes contd

Standard array

For row i 2,3,...,
2 n k
find a vector in Vn of minimum
weight that is not already listed in the array.
Call this pattern e i and form the i : th row as the
corresponding coset
zero
codeword U1 U2 U2k
e2 e2 U2 e2 U2k
coset

coset leaders
e2nk e2nk U2 e2nk U2k

Lecture 9 28
 Linear block codes contd

Standard array and syndrome table decoding

1. Calculate S rHT
2. Find the coset leader, e ei , corresponding to S.
re
3. Calculate U and the corresponding m .

Note that U
r

e
(

U

e)
e
U

(
e
)
If e e , the error is corrected.
If e e , undetectable decoding error occurs.

Lecture 9 29
 Linear block codes contd
Example: Standard array for the (6,3) code
codewords

000000
110100
011010
101110
101001
011101
110011
000111
000001
110101
011011
101111
101000
011100
110010
000110
000010
110111
011000
101100
101011
011111
110001
000101
000100
110011
011100
101010
101101
011010
110111
000110
111100
001000
010000
100100 coset
100000
010100
010001
100101 010110

Coset leaders

Lecture 9 30
 Linear block codes contd

Error pattern Syndrome

000000 000 U(101110)
transmit
ted.
000001 101
r(001110)
is
received.
000010 011
The
syndrome
of
ris
computed
:
000100 110
001000 001 S T
rH T

(001110)
H (100)
010000 010 Error
pattern
correspond
ing
to
this
syndro
is
100000 100

e (100000)
010001 111
The
corrected
vector
is
estimated

U r
e
(001110)
(100000)
(10111

Lecture 9 31
Hamming codes

Hamming codes
Hamming codes are a subclass of linear block codes and
belong to the category of perfect codes.
Hamming codes are expressed as a function of a single
minteger
2 .
Code length : n 2m 1
Number of informatio n bits : k 2 m m 1
Number of parity bits : n-k m
Error correction capability : t 1

The columns of the parity-check matrix, H, consist of all

non-zero binary m-tuples.

Lecture 9 32
 Hamming codes

Example: Systematic Hamming code (7,4)

10 0 0111

H01 0 1 011[
I T
P]
3
3

0 011101

0 1 1 1 0 0 0
1 0 1 0 1 0
0
G [P I 44 ]
1 1 0 0 0 1 0

1 1 1 0 0 0 1

Lecture 9 33
 Cyclic block codes
Cyclic codes are a subclass of linear
block codes.
Encoding and syndrome calculation are
easily performed using feedback shift-
registers.
Hence, relatively long block codes can be
implemented with a reasonable complexity.
BCH and Reed-Solomon codes are cyclic
codes.

Lecture 9 34
 Cyclic block codes

A linear (n,k) code is called a Cyclic code

if all cyclic shifts of a codeword are also
codewords.

U
(
u
0,
u
1,
u i cyclic shifts of U
,...,
2nu
)

1

U
(
u(
i
)
,

n
iu
n

i
1,...,
u
,

n
1u
,
0u
,
u
1,...
2u
n

i
1)
Example:
U (
110 )

(
1
(
) (
U 2
111
)U
(
01
)
) (
3
U
(
10
)
) (
4
)
U
(
1)
U

Lecture 9 35
 Cyclic block codes
Algebraic structure of Cyclic codes, implies expressing
codewords in polynomial form

U
(
X
)
u
0
u
X
2
u
X
12n
1
...
u
X
1 de

n (
n-
1
)
Relationship between a codeword and its cyclic shifts:
n 1
X U (X ) u0X u1X 2
..., u n2 X u n 1 X n

u n 1 u 0 X u 1 X 2 ... u n 2 X n 1 u n 1 X n u n 1

U (1 )
(X ) u n 1 (X n
1)

U (1 )
(X ) u n 1 ( X
n
1)
(
1
)
U
(X
)
X
U
(
X)
mo
(
n
X
1
)
Hence:
By extension (
i
)
U
(X
)
Xi
U(
X) n
mod
(
X
1
)

Lecture 9 36
 Cyclic block codes
Basic properties of Cyclic codes:
Let C be a binary (n,k) linear cyclic code
1. Within the set of code polynomials in C, there
is a unique monic polynomial g ( X ) with
minimal degree r n.g (X )is called the
generator polynomial.
g (
X ) g 0 g1X
...gr X r

1. Every code polynomial U( X ) in C can be

expressed uniquely as U (X ) m (X )g (X)
2. The generator polynomial g ( X ) is a factor of
X n 1

Lecture 9 37
 Cyclic block codes

The orthogonality of G and H in polynomial

form is expressed as g (X )h( X ) X n
. This
1
means h( X ) is also a factor of X n 1

1. The row i,i 1 k, of the generator matrix is

,...,
formed by the coefficients of the " i 1" cyclic
shift of the generator polynomial.
g0 g1 gr 0
g(X)
Xg(X) g0 g1 gr
G

k1 g0 g1 gr
X g(X)
0 g0 g1 gr

Lecture 9 38
 Cyclic block codes

Systematic encoding algorithm for an

(n,k) Cyclic code:
nk
1. Multiply the message polynomial m ( X ) by X

1. Divide the result of Step 1 by the generator

polynomial g ( X ) . Let p( X ) be the reminder.

1. Add to )to form the codeword
nk
p ( X ) X m(X
U( X )

Lecture 9 39
 Cyclic block codes

Example: For the systematic (7,4) Cyclic code

with generator polynomial g (X)
1 X X 3

1. Find the codeword for the message m (

1011 )
n 7, k 4, nk 3
m (1011
) m(X) 1 X2 X3
Xnkm(X) X3m(X) X3(1 X2 X3) X3 X5 X6
DivideXnkm(X) byg(X):
X3 X5 X6 (1 X X2 X3)(1 X X3) 1

quotient
q( X) generator
g( X) remainder
p( X)

Formthecodeword
polynomial
:
U(X) p(X) X3m(X) 1 X3 X5 X6
U (1
0
01 0
1
1)
parity
bits message
bits

Lecture 9 40
 Cyclic block codes
Find the generator and parity check matrices, G and H,
respectively.
g(X) 11 X 0 X 2 1 X3 (g0, g1, g2, g3) (1101
)
1 1 0 1 0 0 0
0 1 1 0 1 0 0
Not in systematic form.
G We do the following:
0 0 1 1 0 1 0
row(1
row(
row

0 0 0 1 1 0 1
row(1
row(
row
ro

1 1 0 1 0 0 0
0
0
1 001011
G
1 1 0 1 0

H
0 10111
0
1 1 1 0 0 1 0


0 01011

1
1 0 1 0 0 0 1
I 33 PT
P I 44

Lecture 9 41
 Cyclic block codes

Syndrome decoding for Cyclic codes:

Received codeword in polynomial form is given by
Received r
(X
)
U(
X)
e(
X) Error
codeword pattern

The syndrome is the remainder obtained by dividing the

received polynomial by the generator polynomial.
r
(
X)
q(
X)g
(
X)
S(
X) Syndrome

With syndrome and Standard array, the error is

estimated.

In Cyclic codes, the size of standard array is considerably

reduced.

Lecture 9 42
 Example of the block codes

PB
8PSK

QPSK

E
b/N
0[dB]
Lecture 9 43