Technical University of Munich

Coding and Cryptography

Prof. Dr.-Ing. Antonia Wachter-Zeh

Channel Coding

Prof. Dr.-Ing. Antonia Wachter-Zeh

SS 2021
30.07.2021

• Write your name and student ID on every sheet. Please have your student card ready for
examination.
• The exam duration is 90 minutes.
• This exam has 7 questions on 22 pages (excluding the cover page).
• Do not write with pencils or red pens.
• You can get full credit if and only if you give reasons for your answer.
• Problem parts that are marked with an =⇒ can be solved independently of previous parts
of the same problem.
• Please remain seated until the end of the exam.
• This is an open book exam, you are allowed to use any material that is in paper form
(scripts, notes, print-outs, etc.) and a non-programmable calculator! Other materials are
not allowed.

SAMPLE SOLUTION
For internal use only:
Question Points Score
Channel Coding Principles 12
Linear Codes 15
Reed-Solomon Codes 19
Cyclic Codes 19
Reed–Muller Codes 10
Concatenated Codes 6
Convolutional Codes 9
Total: 90
 Problem 1: Channel Coding Principles (12 points)

Consider the code C ⊆ Fn3 given by

C = {(00000), (11111), (22222), (11222)}.

=⇒ a) What is the length n, the cardinality |C|, and the minimum distance d of the code C? (2 pt)

Solution: n = 5, |C| = 4.
The code length n is the length of each codeword in the codebook and the cardinality
|C| is the number of codewords.
The distance is d = 2.

=⇒ b) Is the code C linear? Justify your answer. (1 pt)

Solution: No, since, the word 2 · (11222) is not in this code. Or some linear combi-
nations of the codewords are not in this code, e.g., (11111) + (11222) = (22000) ∈
/ C.

=⇒ c) Consider a received word r = c + e, where c ∈ C. Give an example of a non-zero error (2 pt)

vector e that can be corrected uniquely by a nearest codeword decoder in this code.
Can all error vectors of this weight be corrected? Justify your answer!

Solution: Any vector e ∈ {(00100), (00010), (00001), (00200), (00020), (00002), ...}.
Code distance is d = 2, so even for weight 1 not all errors are correctable.

Page 1 - Please turn page

 Consider a transmitter sending the codeword c of C with probability



P c = (00000) = 0.1, P c = (11111) = 0.5



P c = (22222) = P c = (11222) = 0.2

over the q-ary symmetric channel (q = 3) with crossover probability p. The received word
at the channel output is denoted by r.
=⇒ d) First assume p = 0. Calculate the probability that the first symbol of the received word (1 pt)
is 1, i.e., P (r0 = 1).

Solution:



P (r0 = 1) = P c = (11111) + P c = (11222) = 0.7

=⇒ e) Now assume p = 0.1. Calculate the probability that the first symbol of the received (2 pt)
word is 1, i.e., P (r0 = 1).

Solution:



P (r0 = 1) = (1 − p)P c = (11111) + (1 − p)P c = (11222)
p
p

+ P c = (00000) + P c = (22222)
2 2
129
=
200

Page 2 - Please turn page

 f) For which value of the crossover probability p is the probability of receiving any symbol (2 pt)
in any position always the same, i.e., P (r0 = 0) = P (r0 = 1) = P (r0 = 2) = P (r1 =
0) = . . .?
Hint: In a uniformly random vector any value is equally likely in each position. You do
not need to calculate any probabilities!

p
Solution: For p = 23 we have (1−p) = q−1 = 13 , i.e., the channel output is uniformly
distributed and independent of the input. Each symbol occurs with probability 13 .

For the following tasks concerning MAP and ML decoding the crossover probability is fixed
to be p = 0.6. The channel input probabilities are still given by



P c = (00000) = 0.1, P c = (11111) = 0.5



P c = (22222) = P c = (11222) = 0.2 .

=⇒ g) A receiver needs to decide whether to use a MAP or ML decoder. Are these decoders (2 pt)
equivalent in the given setting? Which decoder should the receiver choose to get the
best performance?

Solution: Not equivalent, channel input not uniformly distributed.

MAP decoder is always at least as good as an ML decoder.

Page 3 - Please turn page

 Problem 2: Linear Codes (15 points)

Consider the linear code C over F3 with parity-check matrix

 
2 2 1 1 0 0 0
2 2 1 0 1 0 0
H= 0
 .
1 0 0 0 1 0
0 2 1 0 0 0 1

=⇒ a) Give the length n and dimension k of the code C. (1 pt)

Solution: n = 7 (0.5 pts)

k = 3 (0.5 pts)

=⇒ b) Give a generator matrix G of the code C. (2 pt)

Solution:
 
1 0 0 1 1 0 0
G = 0 1 0 1 1 2 1
0 0 1 2 2 0 2

(Give 1pt if the entries of 2 are replaced by 1 in the answer.)

=⇒ c) The minimum distance of the code C is d ≥ 3. Show that this bound is tight, i.e., d = 3. (2 pt)

Solution: Columns 1, 4, 5 are linearly dependent.

Page 4 - Please turn page

 d) Consider a received word r = c + e. Give an error vector of weight wt(e) = 3 that is (2 pt)
not detectable by the code C.

Solution: Possible choices: (1, 0, 1, 0, 0, 0, 2), (2, 0, 0, 2, 2, 0, 0), (0, 2, 2, 0, 0, 1, 0),

(2, 0, 2, 0, 0, 0, 1), (0, 1, 1, 0, 0, 2, 0), (1, 0, 0, 1, 1, 0, 0)
These error vectors results in an all-zero syndrome and are therefore not detectable.

=⇒ e) Is the code C optimal with respect to the Singleton bound? (1 pt)

Solution: The Singleton bound gives d ≤ n − k + 1 = 5, so the code is not optimal

w.r.t. this bound.

=⇒ f) A word r = c + e = (0, 2, 2, 0, 1, 1, 0) is received, where c ∈ C. Calculate the syndrome (3 pt)

vector s of r corresponding to C and give the error vector e of smallest possible weight
corresponding to this syndrome.

Solution: s = (0, 1, 0, 0).

Syndrome is the fifth column of H, so e = (0, 0, 0, 0, 1, 0, 0)

Page 5 - Please turn page

 Now consider a system that operates over a binary channel, i.e., in the field F2 .
g) Use the sphere-packing bound to determine the maximal dimension of a binary code of (2 pt)
length n = 7 and minimum distance d = 3.

Solution: Sphere-packing bound gives

  
k 7
2 + 1 ≤ 27 ⇒ 2k ≤ 16
1

=⇒ h) Consider a received word r = c + e, where c is a codeword of the [7, 4, 3] Hamming (2 pt)

code and wt(e) = 2. Let ĉ be the output of a nearest codeword decoder applied to r.
Argue why it always holds that ĉ 6= c.

Solution: The Hamming code is perfect, so the Hamming balls of radius 1 fill the
entire space. Since the word r is not in the radius-1 Hamming ball around c, it
must be in the radius-1 Hamming ball around some other codeword c0 , which will
be the output of the nearest codeword decoder.

Page 6 - Please turn page

 Problem 3: Reed-Solomon Codes (19 points)

Let α be a primitive element of Fpm . The elements of Fpm are given in the table below.

Power of α Polynomial representation

− 0
α0 1
α1 x
α2 x2
α3 x+1
α4 x2 + x
α5 2
x +x+1
α6 x2 + 1

=⇒ a) Determine the parameters p and m of this field. Give the irreducible polynomial f (x) (2 pt)
used to construct this field.

Solution: p = 2 (0.5 pts)

m = 3 (0.5 pts)
f (x) = x3 + x + 1 (1pt)

=⇒ b) Give the polynomial representation of multiplicative and additive inverses of α4 and (3 pt)
x2 + 1. Is there an element in Fpm that does not have a multiplicative inverse? Is there
an element that does not have an additive inverse?

Solution: α−4 = αq−1−4 = α3 = x + 1 (0.5 pts)

−α4 = α4 (0.5 pts)
α−6 = α1 = x (0.5 pts)
−α6 = α6 (0.5 pts)
0 does not have a multiplicative inverse. (0.5 pts)
Every element has an additive inverse. (0.5 pts)

Page 7 - Please turn page

 Consider a Reed-Solomon RS code over F8 generated by
 
1 1 1 1 1 1 δ0
GRS = 1 x x2 x + 1 x2 + x x2 + x + 1 δ1  .
1 x2 x2 + x x 2 + 1 x x+1 δ2

=⇒ c) Give a choice for the values of δ0 , δ1 , and δ2 such that the code is an RS code. For your (3 pt)
choice, is the code a primitive RS code?

Solution: The choices are (δ0 , δ1 , δ2 ) = (1, 0, 0) and (δ0 , δ1 , δ2 ) = (1, x2 + 1, x2 + x +

1). The latter gives a primitive RS code.

=⇒ d) Give the length n, dimension k, and minimum distance d of the code RS. (1 pt)

Solution: n = 7, k = 3
d=n−k+1=5

Page 8 - Please turn page

 Now consider the correction of t = 2 errors in this RS code. The received word is given by

r = c + e = (x, x2 , x2 + 1, x2 + x + 1, x2 , 1, x2 )

and the syndrome is given by

s = r · H> 2
RS = (1, 0, x + 1, 1).

=⇒ e) Set up and solve the linear system of equations S · Λ = Ω for the coefficients of the (4 pt)
error locator polynomial Λ(y).

Solution:
     2 
0 1 Λ1 x +1
· =
x2 + 1 0 Λ2 1

This system gives 1 · Λ2 = x2 + 1 and (x2 + 1) · Λ1 = 1. The coefficients of Λ(y) are

therefore Λ1 = x and Λ2 = x2 + 1.

=⇒ f) After decoding, the correct codeword was determined to be (2 pt)

c = (x, x2 , x2 + x + 1, x2 + x + 1, x, 1, x2 ) .

Give the roots of the error locator polynomial Λ(y).

Solution: Comparing the codeword to the received word shows that the errors
occurred in the positions corresponding to the locators x2 = α2 and x2 + x = α4 .
The roots of the ELP are given by the multiplicative inverses of these elements, i.e.,
by α5 and α3 .

Page 9 - Please turn page

 The systematic generator matrix of an RS code over Fq is given by
 
1 0 ··· 0 0 r0,0 r0,1 ··· r0,n−k−1
0 1 · · · 0 0 r1,0 r1,1 ··· r1,n−k−1 
 
G0RS =  ... ... . . . .. .. .. .. .. ..
 ,
 
. . . . . .
 
0 0 · · · 1 0 rk−2,0 rk−2,1 · · · rk−2,n−k−1 
0 0 ··· 0 1 rk−1,0 rk−1,1 · · · rk−1,n−k−1

where the ri,j are elements of Fq .

=⇒ g) Argue why ri,j 6= 0 must hold for all i = 0, . . . , k − 1 and j = 0, . . . , n − k − 1. Is this (4 pt)
also a sufficient condition for the code to be MDS?
Hint: Consider the Hamming weight of a row of this matrix.

Solution: The weight of each row of the systematic generator matrix is 1 plus the
number of non-zero parity symbols in this row. There are n − k parity symbols in
each row and from the Singleton bounds we know that the minimum distance of
this code is n − k + 1. Hence, each non-zero codeword and, specifically, each row
of the generator matrix is of weight ≥ n − k + 1. It follows that every ri,j must be
non-zero.
No, it is not sufficient. Every subset of k columns must be independent, which is
clearly violated if, e.g., all ri,j are chosen to be equal (arguing via the parity-check
matrix is also fine).

Page 10 - Please turn page

 Problem 4: Cyclic Codes (19 points)

Consider a cyclic code over a finite field Fq with q = 5 and length n = 8. The polynomial
x8 − 1 can be factored over F5 into irreducible polynomials as follows

x8 − 1 = (x + 1) · (x + 2) · (x + 3) · (x + 4) · (x2 + 2) · (x2 + 3).

A cyclic code C is constructed by the generator polynomial
g(x) = (x + 1) · (x + 2) · (x + 3) · (x2 + 2) = x5 + x4 + 3x3 + 3x2 + 2x + 2.
=⇒ a) Compute the rate R of the code C. (1 pt)

Solution:
3
The dimension k = n − deg{g(x)} = 8 − 5 = 3 and the rate R = .
8

=⇒ b) Write down the codeword c (in vector representation) that is obtained by encoding (1 pt)
u(x) = 1.

Solution: The information polynomial u(x) = 1 is encoded as c(x) = u(x) · g(x) =

g(x) and then the mapping should be c = (2, 2, 3, 3, 1, 1, 0, 0)

c) Give a generator matrix G of the code C. (2 pt)

Solution: The generator-matrix G can be written if one knows g

 
2 2 3 3 1 1 0 0
G= 0 2 2 3
 3 1 1 0
0 0 2 2 3 3 1 1

Page 11 - Please turn page

=⇒ d) Compute the parity-check polynomial h(x) of C. (1 pt)

Solution:
x8 − 1
The parity-check polynomial h(x) = = (x + 4) · (x2 + 3) = x3 + 4x2 + 3x + 2
g(x)

e) Compute the generator polynomial g ⊥ (x) of the dual code C ⊥ . In this occasion, can the (2 pt)
parity-check matrix of C be a valid generator-matrix G⊥ of the dual code C ⊥ ? Justify
your answer!

Solution: The generator polynomial of the dual code

xk h(x−1 ) x3 · (x−3 + 4x−2 + 3x−1 + 2)

g ⊥ (x) = = = x3 + 4x2 + 2x + 3 6= h(x)
h(0) 2

Even though g ⊥ (x) 6= h(x) the generator matrix G⊥ = H since the codes are dual.

Page 12 - Please turn page

 1
=⇒ f) Prove that for any binary cyclic code of rate R = and length n = 2i for i ∈ N, the (3 pt)
2
minimum distance is d = 2.
Hint: (xn + 1) = (x + 1)n if n is a power of 2.

Solution:
i i i−1 i−1
Over F2 we can write (xn − 1) = (x2 + 1) = (x + 1)2 = (x + 1)2 · (x + 1)2 =
1 n
g(x) · h(x) since the rate is R = and deg{g(x)} = n − k = = 2i−1 . From the
2 2
hint we can write
i−1 i−1
g(x) = (x + 1)2 = x2 + 1
Since g(x) is a codeword then d ≤ wt(g) = 2.
The equality can be shown by saying that g(x) is a minimum weight codeword or
by showing that encoding u(x) into c(x) gives that: wt(c(x)) = 2 · wt(u(x))

We would like to construct a ternary (q = 3) BCH code of length n = 10.

=⇒ g) Find the smallest possible extension degree s such that n divides q s − 1. (1 pt)

Solution: One needs to compute 3s − 1 for s = 1, 2, · · · and can compute that the
smallest possible s s.t. n|(3s − 1) is s = 4.

Page 13 - Please turn page

=⇒ h) Compute all cyclotomic cosets for the given parameters q = 3 and n = 10. (2 pt)

Solution:

C0 = {0}
C1 = {1, 3, 7, 9} = C3 = C7 = C9
C2 = {2, 4, 6, 8} = C4 = C6 = C8
C5 = {5}

i) What are the possible dimensions for a ternary BCH code of length n = 10 ? (2 pt)

Solution: kBCH = n − |D|

|D| ∈ {1, 2, 4, 5, 6, 8, 9} and also trivial (|D| = 0, 10) (if |D| = 0, 10 is not included
it is still fine)
k ∈ {1, 2, 4, 5, 6, 8, 9}

Page 14 - Please turn page

j) Pick the set D such that the BCH bound gives δ = 4. Does there exist another BCH (3 pt)
code of the same rate with a different BCH bound? If yes, which one is better, if not,
explain why.

Solution: In order to have a BCH bound d ≥ δ = 4 we need that the set D

should have δ − 1 = 3 consecutive integers and this can be achieved by picking
1
D = C2 ∪ C5 = {2, 4, 5, 6, 8} and the rate of this code is R = .
2
We can indeed construct 3 more BCH codes of rate half, but the BCH bound will tell
us δ ∈ {2, 3, 4} so we are already maximizing the lower bound with our constraint.
One can also pick D = C0 ∪ C2 ∪ C5 and the rate is changed but the last property
still holds.

k) State the parameters of the code to which our constructed BCH code is a subfield (1 pt)
subcode.

Solution: Our BCH code is a subfield subcode of a RS(n, n − δ + 1, δ) = [10, 7, 4]34

code

Page 15 - Please turn page

 Problem 5: Reed–Muller Codes (10 points)

Consider a high-order Reed-Muller code RM(r, m).

=⇒ a) For r = 3 what is the smallest length Reed-Muller code that can correct t = 3 errors. (3 pt)
Please state the length, dimension and the minimum distance of such a code.

Solution: In order to correct exactly t = 3 errors we need that d = 7 or 8. For RM

codes only d = 8 is possible.
Thus, 2m−3 = 8 then m = 3 + 3 = 6 .
Thus we need a RM code [64, 42, 8]2

b) What are the Reed-Muller component codes Cu and Cv that one can use to construct (2 pt)
the code in a)? Do they both have smaller minimum distance than the code in a) ?

Solution: To construct a RM(3, 6) code one can use as Cu = RM(3, 5) and Cv =

RM(2, 5).
Indeed no, because dC = dCv = 8

=⇒ c) Show that for any valid m, the all-one codeword belongs to the RM(r > 2, m) code. (3 pt)

Solution: We can write the given RM code as RM(r, m) = {(u, u + v) : u ∈

RM(r, m − 1), v ∈ RM(r − 1, m − 1)} and the only way we can have the all-one
codeword is if u = 1 and v = 0. v can be the all-zero codeword since any Reed-
Muller code is linear. The only thing we need to show is that RM(r, m−1) contains
the all-one codeword.
r
If the recursion is done m − r times then we have that RM(r, r) = F22 should
contain the all-one codeword which indeed holds.

Page 16 - Please turn page

d) Write 3 more codewords that any Reed-Muller code RM(r > 0, m) has in its codebook, (2 pt)
(except for the all-one codeword). Justify your answer!

Solution: Due to linearity, every Reed-Muller code contains the all-zero codeword.
Due to the construction RM(r, m) = {(u, u + v) : u ∈ RM(r, m − 1), v ∈
RM(r − 1, m − 1)}, we can have that (0, 1) and (1, 0) by picking u = 0 = v̄

Page 17 - Please turn page

 Problem 6: Concatenated Codes (6 points)

In this problem we want to construct a generalized concatenated code with l = 3 inner and
outer codes. The three binary inner codes are:
ˆ B1 : Single Parity Check Code (nSPC = 8)
ˆ B2 : Reed-Muller Code of first-order (r = 1, m = 3)
ˆ B3 : Repetition Code (n = 8)
=⇒ a) Argue why it holds that B3 ⊂ B2 ⊂ B1 . (2 pt)

Solution:
ˆ B1 : SPC [8, 7, 2]2
ˆ B2 : RM(1, 3) = [8, 4, 4]2
ˆ B3 : Rep. Code(n = 8) = [8, 1, 8]2
B3 has only two codewords, the all-zero and the all-one and both this codewords are
contained in B2 and B1 .
B2 , apart from the codewords mentioned above, has only weight 4 codewords, which
are all contained in B1 since it is a single parity check code.

=⇒ b) Find the parameters of the outer codes A1 , A2 , A3 such that the generalized concate- (4 pt)
nated code is a [72, 41, ≥ 8]2 code and A1 is an MDS code.

Solution: We can achieve our required code if we pick

ˆ A1 = [9, 4, 6]24 → RS code
ˆ A2 = [9, 8, 2]22 → SPC code
ˆ A3 = [9, 9, 1]2 → trivial code
check that k = 41 = 4 · 4 + 8 · 2 + 9 · 1

Page 18 - Please turn page

Solution:

Page 19 - Please turn page

 Problem 7: Convolutional Codes (9 points)

The following shift register encoder of a convolutional code is given.

(i) c(1)

u(1) c(2)

u(2) + c(3)

(ii) c(4)

=⇒ a) What is the memory m and constraint length ν of this convolutional code? What is the (2 pt)
rate R? Is the convolutional encoder (quasi-)systematic? Justify your answer!

Solution: The memory is m = 1 and the contraint length is ν = 2.

k 2 1
The rate is R = = =
n 4 2
The encoder is quasi-sytematic since c(1) = u(1) and c(4) = u(2)

=⇒ b) Compute the polynomial generator matrix G(D). (1 pt)

 
1 D D 0
Solution: G(D) =
0 0 D 1

Page 20 - Please turn page

=⇒ c) What is the transmitted codeword if the information sequences are u(1) = (1, 0, 1), (2 pt)
u(2) = (0, 0, 1) and we terminate the code? What is the rate after termination Rterm ?

Solution: For the given information sequence the transmitted codeword is:

c = (1000|0110|1001|0100).
6 3
The rate Rterm = =
16 8

=⇒ d) Assume that due to manufacturing problems, the lines depicted as (i) and (ii) in the (2 pt)
figure are disconnected. How can we reconstruct the information sequence if error-free
transmission is guaranteed? Is it possible to reconstruct it if only one error occurred?
Justify your answer!

Solution:
ˆ Error-free: û(1) = c(2) · D−1 and û(2) = c(3) · D−1 − û(1) where
D−1 is inverting the unit-delay.
ˆ if wt(e) = 1, we cannot reconstruct the information sequence anymore since the
redundancy is zero.

Page 21 - Please turn page

 Consider the following part of the trellis of a Viterbi decoder. Assume that up to now, all
the incoming paths have the same path-metric. The receiver knows that a binary erasure
channel with erasure probability  = 0.6 was used for the transmission.

r = (1 0 ? ? ?)

00000
S0t S0t+1
1
1010

S1t 11
011

111
11
S2t

S3t

Assume that r is the received sub-sequence at the step t of decoding. The erasure symbol
is denoted by ?.
=⇒ e) Which of the path(s) will not be removed? (2 pt)

Solution: Since for a BEC we have that Pr{r = 1/x = 0} = Pr{r = 0/x = 1} = 0
only the path whose output starts with 10 · · · will be considered. Thus :
S0t S0t+1
1
1010

S1t

S2t

S3t

Only the path depicted in red can survive.

Page 22 - End of exam

End of exam - extra page

Solution:

Page 23 - Please turn page