ECE458

INFORMATION THEORY AND CODING

LECTURE 9
Associate Prof. Fatma Newagy
Fatma_newagy@eng.asu.edu.eg
LAST LECTURE TOPICS
 Channel coding
 Linear Systematic Block Codes
Generator Matrix and Encoding Process

fatma_newagy@eng.asu.edu.eg


 Parity Check Matrix

 Encoding Circuit

2
TODAY’S TOPICS
 Syndrome
 Error Detection and Syndrome Circuit
Hamming weight and Hamming distance

fatma_newagy@eng.asu.edu.eg


 Error Correction Capability

 Hamming Code
 Dual Codes

3
 SYNDROME
 Characteristic of parity check matrix (H)

T
v.H  0 vC

fatma_newagy@eng.asu.edu.eg
T
v.H  0 vC
v Channel r
Syndrome
v + r=v+e
T
s=r.H
e Error Pattern
4
 ERROR DETECTION

s0 rC
r is NOT a codeword

fatma_newagy@eng.asu.edu.eg
An Error is Detected: What Options do we have?

 Ask for Retransmission of Block

 Automatic Repeat Request (ARQ)
 Attempt the Correction of Block
 Forward Error Correction

5
 UNDETECTABLE ERROR PATTERNS

s0 rC

fatma_newagy@eng.asu.edu.eg
 Can we be sure that r=v ??
 NO! WHY?

eC  r C
 How many undetectable error patterns exist?
 2k-1 Nonzero codeword means
2k-1 undetectable error patterns
6
SYNDROME DECODING

1. Compute syndrome of r (i.e., rHT)

2. Locate el whose syndrome is equal to rHT
3. v*= r+el where v* is the codeword output by the

fatma_newagy@eng.asu.edu.eg
decoder

7
SYNDROME CIRCUIT

r0 r1 r2 r3 r4 r5 r6

fatma_newagy@eng.asu.edu.eg
+ + +
1 0 0
0 1 0
 
s0 s1 s2 0 0 1
HT  1 1 0

s 0  r 0  r3  r5  r 6
0 1 1
s 1  r1  r3  r 4  r5  
1 1 1
s 2  r 2  r 4  r5  r 6 1
8
0 1 
HAMMING WEIGHT AND HAMMING DISTANCE
Hamming Weight w(v):
The number of nonzero components of v

fatma_newagy@eng.asu.edu.eg
Hamming Distance d(v,w):
The number of places where v and w differ
Example:
 v= (1 0 1 1 1 0 1): w(v)=5
 w=(0 1 1 0 1 0 1): d(v,w)=3

Important Remark: In GF(2)

d(v,w)=w(v+w)
9

In the example above v+w=(1 1 0 1 0 0 0)

 MINIMUM DISTANCE OF A BLOCK CODE
Minimum Distance of a block code (dmin):
The minimum Hamming distance between any two
code words in the code book of the block code

dmin min d  v, w  : v, w  C, v  w

For Linear Block Codes

dmin  min w  v  w  : v, w  C, v  w
dmin  min w  x  : x  C, x  0
dmin wmin (minimum weight of the code) 10
THEOREM
Let C be an (n,k) linear block code with parity check
matrix H. For each code word of Hamming weight l,
there exists l columns of H such that the vector sum of
these l columns is equal to the zero vector.

fatma_newagy@eng.asu.edu.eg
PROOF:
H  h0 h1 . . hn-1  ,hi is the ith column
Let v   v 0 v1 . . v n-1  be a codeword of weight l
By Definition :
vHT  0  v 0 h0  v1 h1  ...  v n-1 hn-1  0
Assume v i1 , v i2 ,..., v il are the l nonzero components of v
11
 hi1  hi2  ...  hil  0
THEOREM (CONT’D)
Conversely, if there exists l columns of H whose
vector sum is the zero vector, there exists a
codeword of Hamming weight l in C.

fatma_newagy@eng.asu.edu.eg
PROOF:
Suppose hi1 ,hi2 ,...,hil are l columns of H such that :
hi1  hi2  ...  hil  0
For a binary n - tuple x with l nonzero component x i1 , x i2 ,..., x il
xHT  x 0 h0  x1 h1  ...  x n-1 hn-1
xHT  hi1  hi2  ...  hil  0
 x is a valid codeword 12
COROLLARIES

Given C is a linear block code with parity check

matrix H. If no d-1 or fewer columns of H add to 0,
the code has minimum weight of at least d.

fatma_newagy@eng.asu.edu.eg
Given C is a linear block code with parity check
matrix H. The minimum distance of C is equal to the
smallest number of columns of H that sum to 0.

13
 EXAMPLE
(7,4) Linear Block Code

1 0 0 1 0 1 1
H= 0 1 0 1 1 1 0
0 0 1 0 1 1 1

 No all 0 column. No two columns are identical:

 dmin>2
 Columns 0, 1, 3 sum to zero vector
 dmin=3

14
 ERROR CORRECTION CAPABILITY
Error Detection Capability means:
HOW MANY ERRORS ARE GUARANTEED TO BE DETECTED AT
THE RECEIVER SIDE

fatma_newagy@eng.asu.edu.eg
TWO Possibilities:

Number of errors Number of errors

v Channel r (no. of 1s in e) is
e
v: transmitted codeword
e: error pattern caused by
channel
r: received pattern
If error pattern includes l
errors: d(v,r)=l
(no. of 1s in e) is
smaller than dmin greater than dmin
You could be There is a chance
absolutely sure that r is a valid
that r is not a codeword (if e is a
valid codeword. valid code word)
THE ERROR IS THE ERROR
GUARANTEED TO PATTERN MIGHT 15
BE DETECTED NOT BE DETECTED
ERROR DETECTION CAPABILITY

The “Error Detection Capability” of a code defines

the number of errors that are GUARANTEED to
be detected.

fatma_newagy@eng.asu.edu.eg
For a code with minimum Hamming distance dmin:

The error detection capability is dmin-1

16
NUMBER OF DETECTABLE ERROR PATTERNS

An error pattern is defined as any nonzero n-tuple

that could affect a transmitted code word.

fatma_newagy@eng.asu.edu.eg
For an (n,k) code:
 There are 2n-1 error patterns. Note that (0 0 0 . . . 0) is
not an error pattern
 There are 2k-1 undetectable error patterns. Note that
(0 0 0 . . . 0) is selected to be a valid codeword
An (n,k) code has the ability to detect 2n-2k error patterns
Example: A (7,4) code is able to detect:
128-16=112 error patterns

17
 ERROR CORRECTION CAPABILITY

The “error correction capability” defines the number

of errors that are GUARANTEED to be corrected.

fatma_newagy@eng.asu.edu.eg
For a code with minimum Hamming distance dmin:
The error correction capability is (dmin-1)/2
x means floor(x)

18
HAMMING CODES
For any positive integer m≥3, there exists a
Hamming code such that:

fatma_newagy@eng.asu.edu.eg
- Code Length: n = 2m-1
- No. of information symbols: k = n-m =2m-m-1
- No. of parity check symbols: n-k = m
- Error correcting capability: t = 1 (i.e., dmin=3)

19
 PARITY CHECK MATRIX, GENERATOR
MATRIX AND DMIN OF HAMMING CODES

H contains all of nonzero m-tuples as its columns. In

the systematic form:

m columns 2m-m-1 columns of

fatma_newagy@eng.asu.edu.eg
of weight 1 weight >1

H = [Im Q] mxk
G = QT [ I2m -m-1
K
]
Parity Check Matrix Generator Matrix

- No two columns are identical dmin>2

- The sum of any two columns must be a third one dmin=3
- Hamming Codes can correct all single errors or detect all double
errors
20
EXAMPLE: (15,11) HAMMING CODES

- Code Length: n = 24-1 = 15

- No. of information symbols: k = 15-4 = 11
- No. of parity check symbols: 15-11 =4

fatma_newagy@eng.asu.edu.eg
- Error correcting capability: dmin =3

4 columns of 11 columns of
weight 1 weight >1

 1 0 0 0 1 1 1 0 0 0 1 1 1 0 1 
 
0 1 0 0 1 0 0 1 1 0 1 1 0 1 1
H 
 0 0 1 0 0 1 0 1 0 1 1 0 1 1 1 
 
 0 0 0 1 0 0 1 0 1 1 0 1 1 1 1  21
DUAL CODES
 Every (n,k) linear block code with generator matrix 𝑮 and
parity check matrix 𝑯 has a dual code (𝑛,𝑛−𝑘) with
generator matrix 𝑯 and parity check matrix 𝑮

fatma_newagy@eng.asu.edu.eg
22