Information Theory and Coding

EC401 M1-Information Theory & Coding
Module 6
KTUStudents.in
ktustudents.in
For more study materials: WWW.KTUSTUDENTS.IN

Syllabus
• Convolutional Codes
• Encoding
• Time and frequency domain approach
• KTUStudents.in
State , tree and trellis diagram
• Transfer function and minimum free distance
• Maximum likelihood decoding
• Viterbi Algorithm
• Sequential Coding

Convolutional coding
• Type of Channel Coding
KTUStudents.in

• What is difference between block codes, cyclic
codes and convolutional codes
• To get high redundancy -memory locations
KTUStudents.in
should more-digital communication s/m will
complex. Block Codes and Cyclic
Codes
• Using Less number of memory locations-high
redundancy can be achieved-complexity will
be less Convolutional coding

• (n,k,m ) Convolutional Encoder
KTUStudents.in

Convolutional codes
KTUStudents.in
k input
(n,k,m)
n output
bits bits
m bits in memory

CONVOLUTIONAL CODES
• An (n,k,m) convolutional code can be
implemented with a k-input, n-output linear
sequential circuit with input memory m.
KTUStudents.in
• Convolutional code differs from block codes in
that
– Encoder contains memory
– At any given time the n-outputs depends on the k-
inputs at that time and m previous input blocks.
• n and k are small integers with k<n
• m must be large for low error probability.
Example 1: (2,1,3) Convolutional
Encoder
KTUStudents.in
Implemented by Linear feed forward Shift register
are generator sequences

Encoding Equations
KTUStudents.in

• Encoding equations: denotes
discrete convolution
• Convolution operation: For any l ≥ 0 and
KTUStudents.in
• For example 1,
• The codeword is given by,

Example 1
KTUStudents.in

Generator Matrix of (n,k,m)
Convolutional Encoder
KTUStudents.in

Example 1
KTUStudents.in

Transform Domain Representation
• convolution operations in time domain can be
replaced by more convenient polynomial
multiplications in transform domain.
KTUStudents.in
• For (2,1,m) Convolutional Encoder

KTUStudents.in
In general

For Example 1
KTUStudents.in

Encoder
KTUStudents.in

Encoder
KTUStudents.in

Encoder
KTUStudents.in

Encoder
KTUStudents.in

Question
• Draw (3,1,2) Convolutional Encoder
Assume G(D) = (1+D,1+D2,1+D+D2)
KTUStudents.in

Rate and Constraint Length of
Convolutional Code
• Rate of Convolutional Encoder = k/n
• Constraint Length (K) is defined as the number
of shifts over which a single message bit can
KTUStudents.in
influence the encoder output.
• That is K shifts are required for a message bit
to enter the shift register and finally come
out.
• K=m+1

Question
• Draw ½ rate efficiency and constraint length 4
convolutional encoder: Example 1
KTUStudents.in

State Diagram
• Preparation of State Transition Table
KTUStudents.in

State Diagram Representation
(2,1,3) Convolutional Encoder
Example 1
KTUStudents.in

Question
Consider a (2,1,2) Convolutional code with
KTUStudents.in
– Draw the encoder diagram.
– Draw the state diagram

Code Tree or Tree Diagram
• The Tree Diagram adds the dimension of time
to state diagram.
• Code Tree can be explained with the help of
KTUStudents.in
following Example
• Draw Convolutional Encoder with constraint
length 3 and rate ½ with g(1)=(1 1 1) and g(2)=
(1 0 1). Get state Transition Table.

Draw Convolutional Encoder with
constraint length 3 and rate ½ with
g(1)=(1 1 1) and g(2)= (1 0 1).Example 2
KTUStudents.in

Convolutional Encoder with constraint
length 3 and rate ½. Example 2
Input bit Present State Next State Output Bits
m0 m1 m0 m1 V(1) V(2)
0 0 0 0 0 s0 0 0
1 0 0 1 0 s1 1 1
KTUStudents.in
0 1 0 0 1 s2 1 0
1 1 0 1 1 s3 0 1
0 0 1 0 0 1 1
1 0 1 1 0 0 0
0 1 1 0 1 0 1
1 1 1 1 1 1 0

KTUStudents.in

KTUStudents.in
Message: 11011
Code Word:
11 01 01 00 01

• Input 0 specifies the upper branch and input 1
specifies lower branch.
• Output binary symbols are indicated on the
KTUStudents.in
branch.
• A specific path in the tree is traced from left to
right in accordance with the input message
sequence.
• After first m+1 branches, tree becomes
repetitive.

TRELLIS DIAGRAM
• Trellis Diagram is the extension of state
diagram in time.
KTUStudents.in

Trellis Diagram
KTUStudents.in

Trellis Diagram
• The Trellis contains L+K levels, where L is the length
of the incoming message sequence, and K is
constraint length of the code.
KTUStudents.in
• The levels of Trellis are labeled as j=0,1,……,L+K-1
• Input 0 is drawn as a solid line. Input 1 is drawn as a
dashed line.
• For message, 10011 output sequences 11 10 11 11
01.

Maximum Likelihood Decoding
• Consider code word c is generated from message
u.
• At the receiver side r is received. From r , c and u
are to be estimated.
KTUStudents.in
• Take p(r/c) conditional probability of receiving r
given that v was sent.
• The log likelihood function equals log p(r/c)
• Maximum Likelihood decoder choose the
estimate of c for which the log likelihood function
log p(r/c) is maximum.

For BSC,
KTUStudents.in

• N log(1-p) is constant for all c.
• Maximum Likelihood decoding rule for BSC,
can be restated as choose estimate of c
KTUStudents.in
minimizes the hamming distance between
received vector r and transmitted vector c.

Viterbi Algorithm For BSC
 It is maximum likelihood decoding algorithm
 For BSC, decoding rule is based on minimum hamming distance.
Step 1: Starting at level j=0, compute the partial hamming distance
received code word and the code word received for the single path
entering each node. Store the path and its hamming distance for
KTUStudents.in
each state.
Step 2:Increment the level j by 1.Compute the partial hamming
distance for all paths entering a state by adding the branch
hamming distance entering that state to the hamming distance of
the connecting states at the preceding time unit. For each state,
store the path with the smallest hamming distance, together with
the hamming distance and eliminate all other paths.
Step 3: If j< L+m repeat step 2.Other wise stop.

KTUStudents.in

KTUStudents.in

Minimum Free Distance
• The free distance (dfree) of a convolutional code is
defined as the minimum Hamming distance
between any two code words in the code.
KTUStudents.in
• A convolutional code with free distance dfree can
correct t errors if and only if dfree is greater than
2t.
• If the errors are more than above condition, then
Viterbi algorithm will break down in its operation
resulting same hamming distance for many
number of paths.

Viterbi Algorithm For Non BSC
 For Non BSC, decoding rule is based on maximum likelihood
function log p(r/v)
 M(r/v) = log p(r/v)
Step 1: Starting at level j=0, compute the partial metric received code
word and the code word received for the single path entering each
KTUStudents.in
node. Store the path and its metric for each state.
Step 2:Increment the level j by 1.Compute the partial metric for all
paths entering a state by adding the branch metric entering that
state to the metric of the connecting states at the preceding time
unit. For each state, store the path with the largest metric, together
with the metric and eliminate all other paths.
Step 3: If j< L+m repeat step 2.Other wise stop.

KTUStudents.in
b) log p(ri/vi) table c) C2 log p(ri/vi)+C1 C2=7.195, C1 = 2.3

KTUStudents.in

KTUStudents.in

KTUStudents.in

KTUStudents.in

KTUStudents.in

Transfer Function: Example 2
a: 00
b: 10
c: 01
d:11
KTUStudents.in

• Using Masons Gain formulae
KTUStudents.in
dfree = 5, exponent of first term or smallest

exponent
KTUStudents.in
Instead of X, D. Also L can be appended as done in the previous case.

dfree = 6

Sequential Coding
• A drawback of viterbi algorithm is that it need to
perform fixed number of computation (2K)
irrespective of the noise.
• Sequential Decoding is a decoding procedure
KTUStudents.in
whose decoding procedure is adaptable to the
noise level.
• The major drawback of sequential decoding is
that noisy frames take large amounts of
computation, and decoding time will exceed the
upper limit, causing information to be lost or
erased.

Sequential Coding
The Viterbi metric can be modified as Fano Metric
KTUStudents.in
Where R is code rate.
n is number of output bits

l is number of branches.
When comparing paths of different lengths the path with the largest Fano
metric is the bestFor
path.
more study materials: WWW.KTUSTUDENTS.IN
• Example: Consider (3,1,2) convolutional encoder.
Assume that a codeword is transmitted from this
code over BSC. Received vector:
For BSC is taken as the metric and the
KTUStudents.in
path with the smallest metric is selected as the
maximum likelihood path.
Now consider partial path metric for two paths
different lengths.
eg:
The partial path metric are:

Indicating is the better of two paths. But
we can notice that is more likely to be part
of the maximum likelihood path than .
KTUStudents.in

• Ex: For a BSC with transition probability p, the
Fano metric for the truncated codeword and
are given by,
KTUStudents.in
This indicates that is the better of the two

paths.

Sequential Coding
• Two Approaches for Sequential Coding
• Stack Algorithm
• Fano Algorithm
KTUStudents.in

Stack Algorithm
KTUStudents.in

Stack Algorithm
• Step 1: Load the stack with the origin node in the
tree, whose metric is taken to be zero.
• Step 2:Compute the metric of the successors of
the top path in the stack.
KTUStudents.in
• Step 3:Delete the top path from the Stack.
• Step 4:Insert new paths in the stack and
rearrange the stack in the order of decreasing
metric values.
• Step 5:If the top path in the stack ends at
terminal node in the tree, stop. Otherwise return
to Step 2.
Fano Algorithm
KTUStudents.in

Fano Algorithm
• Decoder starts at origin node with T=0 and M=0.
• It then looks forward to the best 2k succeeding
nodes.
KTUStudents.in
• If MF>T, decoder moves forward and checking
whether the end of the tree has been reached, a
Threshold tightening is done if this node is being
visited for the first time.
• Threshold Tightening is increasing of T to some
extent.
Fano Algorithm
• If MF<t, then the decoder looks backward to
the preceding node.
• If MB <T, then T is lowered and look forward to
KTUStudents.in
the best node.
• If MB>T, the decoder moves back to previous
node.

Information Theory and Coding

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Information Theory and Coding

Uploaded by

Copyright:

Available Formats

EC401 M1-Information Theory & Coding

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

Implemented by Linear feed forward Shift register

are generator sequences

For more study materials: WWW.KTUSTUDENTS.IN

• Convolution operation: For any l ≥ 0 and

• The codeword is given by,

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

b) log p(ri/vi) table c) C2 log p(ri/vi)+C1 C2=7.195, C1 = 2.3

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

dfree = 5, exponent of first term or smallest

Instead of X, D. Also L can be appended as done in the previous case.

For more study materials: WWW.KTUSTUDENTS.IN

For more study materials: WWW.KTUSTUDENTS.IN

n is number of output bits

For BSC is taken as the metric and the

The partial path metric are:

For more study materials: WWW.KTUSTUDENTS.IN

This indicates that is the better of the two