Soft Decision

Soft Decision Decoding

DecodingAlgorithms
Algorithms of
of
Reed-Solomon Codes
Reed-Solomon Codes

Jing Jiang and Krishna R. Narayanan

Department of Electrical Engineering
Texas A&M University
Historical Review of Reed Solomon Codes

Date of birth: 40 years ago (Reed and Solomon 1960)

Related to non-binary BCH codes (Gorenstein and Zierler 1961)

Efficient decoder: not until 6 years later (Berlekamp 1967)

Linear feedback shift register (LFSR) interpretation (Massey 1969)

 Other algebraic hard decision decoder:
Euclid’s Algorithm (Sugiyama et al. 1975)

Frequency-domain decoding (Gore 1973 and Blahut 1979)

Wide Range of Applications of Reed Solomon Codes

NASA Deep Space: CC + RS(255, 223, 32)

Multimedia Storage:
CD: RS(32, 28, 4), RS(28, 24, 4) with interleaving

DVD: RS(208, 192, 16), RS(182, 172, 10) product code

Digitial Video Broadcasting: DVB-T CC + RS(204, 188)

Magnetic Recording: RS(255,239) etc. (nested RS code)

Basic Properties of Reed Solomon Codes

RS(N, K) defined over GF (q m ), where N  q m  1

(1) Polynomial evaluation form : c( x)  ( x1 , x2 ,..., x N )  ( f ( 0 ), f (1 ),..., f ( N 1 ))

where f ( x)  f 0  f1 x  ...  f K 1 x K 1
 i s are distinct nonzero elements in GF (q m )
b  2 t 1
(2) Generator polynomial form : c( x)  g ( x)m( x), where g ( x)   ( x
i b
  i
)

1 b   ( N 1)b 
 
The parity check matrix : H      
1  b  2t 1   ( N 1)( b  2t 1) 
 
Basic Properties of Reed Solomon Codes (cont’d)

 Properties of RS code:

 Symbol level cyclic (nonbinary BCH codes)

 Maximum distance separable (symbol level): d min  2t  1  N  K  1

 Properties of BM algorithm:

 Decoding region: 2e  f  d min

Decoding complexity: o( N 2 ) Usually  d min

2

Motivation for RS Soft Decision Decoder
source interleaving

RS Encoder Π PR Encoder

+
interleaving
AWGN
extrinsic a priori Σ
+
Σ Π +

-
sink de-interleaving
Channel
RS
Decoder a priori
Π 1 extrinsic
Equalizer
hard decision

RS Coded Turbo Equalization System

 Hard decision decoder does not fully exploit the decoding capability

 Efficient soft decision decoding of RS codes remains an open problem

 Soft input soft output (SISO) algorithm is favorable

Presentation Outline

 Symbol-level algebraic soft decision decoding

 Binary expansion of RS codes and soft decoding algorithms

 Iterative decoding for RS codes

 Simulation results

 Applications and future works

Symbol-level Algebraic Soft Decision Decoding
Reliability Assisted Hard Decision Decoding

 Generalized Minimum Distance (GMD) Decoding (Forney 1966):

New distance measure: generalized minimum distance
Successively erase the least reliable symbols and run the hard decision decoder

GMD is shown to be asymptotically optimal

 Chase Type-II decoding (Chase 1972):

Exhaustively flip the least reliable symbols and run the hard decision decoder
Chase algorithm is also shown to be asymptotically optimal

 Related works:
Fast GMD (Koetter 1996)
Efficient Chase (Kamiya 2001)

Combined Chase and GMD for RS codes (Tang et al. 2001)

Performance analysis of these algorithms for RS codes seems still open

Algebraic Beyond Half dmin List Decoding

 Bounded distance + 1 decoding (Berlekamp 1996)

 Beyond d min / 2 decoding for low rate RS codes (Sudan 1997)

 Decoding up N  NK  errors (Guruswami and Sudan 1999)

 A good tutorial paper (JPL Report, McEliece 2003)

Outline of Algebraic Beyond Half Distance Decoding

 Decoding: Transmitte d codeword : ( f (1 ), f ( 2 ),..., f ( N ))

Received vector : ( 1 ,  2 , ... ,  N )
 Basic idea: find f(x), which fits as many points in ( f ( i ),  i ) pairs
 Interpolation Step:
Construct a bivariate polynomial Q ( x, y ) of minimum (1,K-1) degree,
which has a zero of order m at ( l ,  l ), l  1,..., N , i.e.:
if Q( x   l , y   l ) involves no term of degree less than i  j  m
 Factorization Step:
 generate a list of y-roots, i.e.:
L  { f ( x)  F [ x] : ( y  f ( x)) | Q( x, y ), deg( f ( x))  K }
 Pick up the most likely codeword fˆ ( x) from the list L
Complexity:
2 4
Interpolation (Koetter’s fast algorithm): O ( N m )

Factorization (Roth and Ruckenstein’s algorithm):O (NK )

Algebraic Soft Interpolation Based List Decoding

Koetter and Vardy algorithm (Koetter & Vardy 2003)

Based on the Guruswami and Sudan’s algebraic list decoding
Use the reliability information to assign multiplicities

KV is optimal in multiplicity assignment for long RS codes

Reduced complexity KV (Gross et al. submitted 2003)

Re-encoding technique: largely reduce the cost for high rate codes

VLSI architecture (Ahmed et al. submitted 2003)

Soft Interpolation Based Decoding

 Definition:
Reliability matrix:  q N

 Multiplicity matrix: M  g ( )
 Score: S M (c )  M , c
q n
 m  1
 Cost: C ( M )    i , j 
i 1 j 1  2 
 Basic idea: interpolating more symbols using the soft information
 The interpolation and factorization is the same as GS algorithm
 Sufficient condition for successful decoding: S M (c)  2( K  1)C ( M )
2
 The complexity increases with C (M ) , maximum number of multiplicity
Recent Works and Remarks

 Recent works on performance analysis and multiplicity assignment:

 Gaussian approximation (Parvaresh and Vardy 2003)

 Exponential bound (Ratnakar and Koetter 2004)

Chernoff bound (El-Khamy and McEliece 2004)

Performance analysis over BEC and BSC (Jiang and Narayanan 2005)

The ultimate gain of algebraic soft decoding (ASD) over AWGN

channel is about 1dB

 Complexity is scalable but prohibitively huge for large multiplicity

The failure pattern of ASD algorithm and optimal multiplicity

assignment scheme is of interest
Performance Analysis of ASD over Discrete Alphabet Channels

Performance Analysis over BEC and BSC (Jiang and

Narayanan, accepted by ISIT2005)

 The analysis gives some intuition about the decoding radius of ASD

 We investigate the bit-level decoding radius for high rate codes

 For BEC, bit-level radius is twice as large as that of the BM algorithm

 For BSC, bit-level radius is slightly larger than that of the BM algorithm

 In conclusion, ASD is limited by its algebraic engine

 Binary Image Expansion of RS
Codes and Soft Decision Decoding
Binary Image Expansion of RS Codes over GF(2m)

It is known that x  GF (2 m ) can be expressed as an m - dim binary vector xb

Consequently, RS(N, K) code over GF (2 m ) has a binary expansion RSb (Nm, Km)

C1 N  [c0 , c1 ,..., c N 1 ] Cb (1 Nm)  [c0( 0 ) , c0(1) ,..., c0( m 1) ,..., c N( 0)1 , c N(1)1 ,..., c N( m11) ]

 H 0,0 H 0,1  H 0, N 1   h0,0 h0,1  h0, Nm 1 

 
H ( N  K )N     
 H ( N  K 1),0 H ( N  K 1),1  H ( N  K 1),N 
 
 
H b ( N  K ) m Nm     
h( N  K ) m1,0 h( N  K ) m 1,1  h( N  K ) m 1, Nm 1 
 
Bit-level Weight Enumerator

“The major drawback with RS codes (for satellite use) is that the present
generation of decoders do not make full use of bit-based soft decision
information” (Berlekamp)

 How does the binary expansion of RS codes perform under ML decoding?

 Performance analysis using its weight enumerator

 Averaged ensemble weight enumerator of RS codes (Retter 1991)

 It gives some idea about how RS codes perform under ML decoding

Performance Comparison of RS(255,239)
Performance Comparison of RS(255,127)
Remarks

 RS codes themselves are good code

 However, ML decoding is NP-hard (Guruswami and Vardy 2004)

 Are there sub-optimal decoding algorithms using the binary expansions?

Trellis based Decoding using BCH Subcode Expansion

 Maximum-likelihood decoding and variations:

Partition RS codes into BCH subcodes and glue vectors (Vardy and Be’ery 1991)

Reduced complexity version (Ponnampalam and Vucetic 2002)

Soft input soft output version (Ponnampalam and Grant 2003)

Subfield Subcode Decomposition

 B 0 0 0 
 0 B 0 0 
~  BCH subcodes
Gb   0 0 B 0 
 
 0 0 0 B 
Gglue1 Gglue 2 Gglue 3 Gglue 4  Glue vector


 Remarks:
Decomposition greatly reduces the trellis size for short codes

Impractical for long codes, since the size of the glue vectors is very large

 Related work:
Construct sparse representation for iterative decoding (Milenkovic and Vasic
2004)

 Subspace subcode of Reed Solomon codes (Hattori et al. 1998)

Reliability based Ordered Statistic Decoding

 Reliability based decoding:

Ordered Statistic Decoding (OSD) (Fossorier and Lin 1995)
Box and Match Algorithm (BMA) (Valembois and Fossorier 2004)

Ordered Statistic Decoding using preprocessing (Wu et al. 2004)

 Basic ideas:
Order the received bits according to their reliabilities
Propose hard decision reprocessing based on the most reliable basis (MRB)

 Remarks:
The reliability based scheme is efficient for short to medium length codes
The complexity increases exponentially with the reprocessing order

BMA algorithm trade memory for time complexity

Iterative Decoding Algorithms for RS Codes
A Quick Question

How does the panacea of modern communication, iterative

decoding algorithm work for RS codes?

Note that all the codes in the literature, for which we can use
soft decoding algorithms are sparse graph codes with small
constraint length.
How does standard message passing algorithm work?

erased bits

bit nodes
…………. ………..

......... ……………. …………….

? check nodes
If two or more of the incoming messages are erasures the check is erased
For the AWGN channel, two or more unreliable messages invalidate the check
A Few Unreliable Bits “Saturate” the Non-sparse Parity Check Matrix

 Consider RS(7, 5) over GF(23) :

cb   0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

r   0.8 0.9 1.6 0.7 1.0 1.1 0.8 0.4 0.3  0.1 0.5 0.9  0.2  0.1 0.6 0.9  0.2 0.8  0.1 0.5 1.1

1 0 0 0 0 1 0 1 0 1 0 1 0 1 1 1 1 1 1 1 0
0 1 0 1 0 1 0 1 1 1 1 1 1 1 0 1 0 0 0 0 1

0 0 1 0 1 0 1 0 1 0 1 1 1 1 1 1 1 0 1 0 0
Hb   
1 0 0 0 1 0 0 1 1 1 1 0 0 0 1 1 0 1 1 1 1
0 1 0 0 1 1 1 1 0 0 0 1 1 0 1 1 1 1 1 0 0
 
0 0 1 1 0 1 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0
Binary image expansion of the parity check matrix of RS(7, 5) over GF(2 3)

Iterative decoding is stuck due to only a few unreliable bits

“saturating” the whole non-sparse parity check matrix
Sparse Parity Check Matrices for RS Codes

 Can we find an equivalent binary parity check matrix that is sparse?

 For RS codes, this is not possible!

 The H matrix is the G matrix of the dual code

 The dual of an RS code is also an MDS Code

 Each row has weight at least (K+1)

 Typically, the row weight is much higher

Iterative Decoding for RS Codes

 Iterative decoding for general linear block codes:

Iterative decoding for general linear block codes (Hagenauer et al.
1996)

 APP decoding using minimum weight parity checks (Lucas et al. 1998)

 Generalized belief propagation (Yedidia et al. 2000)

 Recent progress on RS codes:
 Sub-trellis based iterative decoding (Ungerboeck 2003)
Stochastic shifting based iterative decoding (Jiang and Narayanan,
2004)
 Sparse representation of RS codes using GFFT (Yedidia, 2004)
Recent Iterative Techniques

 Sub-trellis based iterative decoding (Ungerboeck 2003)

 Self concatenation using sub-trellis constructed from the parity check matrix:

1 0 0 0 0 1 0 1 0 1 0 1 0 1 1 1 1 1 1 1 0
0 1 0 1 0 1 0 1 1 1 1 1 1 1 0 1 0 0 0 0 1

0 0 1 0 1 0 1 0 1 0 1 1 1 1 1 1 1 0 1 0 0
Hb   
1 0 0 0 1 0 0 1 1 1 1 0 0 0 1 1 0 1 1 1 1
0 1 0 0 1 1 1 1 0 0 0 1 1 0 1 1 1 1 1 0 0
 
0 0 1 1 0 1 1 1 1 1 0 0 0 1 0 0 1 1 1 1 0
Binary image expansion of the parity check matrix of RS(7, 5) over GF(2 3)

 Remarks:
Performance deteriorates due to large number of short cycles

Work for short codes with small minimum distances

Recent Iterative Techniques (cont’d)

 Stochastic shifting based iterative decoding (Jiang and Narayanan, 2004)

 Due to the irregularity in the H matrix, iterative decoding favors some bits

 Taking advantage of the cyclic structure of RS codes

[ r05 , r16,, rr20,, rr31 , r42 , r53 , r64 ] Shift by 2
1 1 1 1 0 0 0
H 
1 1 0 0 1 1 0


1 0 0 1 1 0 1

 Stochastic shift prevent iterative procedure from getting stuck

 Best result: RS(63,55) about 0.5dB gain from HDD

 However, for long codes, the performance deteriorates

Proposed Iterative Decoding for RS Codes
Iterative Decoding Based on Adaptive Parity Check Matrix

Idea: reduce the sub-matrix corresponding to unreliable bits to a

sparse nature using Gaussian elimination

 For example, consider (7,4) Hamming code:

transmitted codeword c  0 1 0 1 1 0 0
received vector r  1.1  0.6  0.1  1.4  1.2  0.1  0.1

1 1
0 11 10 11 00 01 0
parity check matrix H 
11 10 01 00 10 11 0
1 


1 1
0 01 01 10 11 00 1 

We can make the (n-k) less reliable positions sparse!

Adaptive Decoding Procedure

unreliable bits

bit nodes
…………. ………..

......... ……………. …………….

check nodes
 After the adaptive update, iterative decoding can proceed
Gradient Descent and Adaptive Potential Function

 Geometric interpretation (suggested by Ralf Koetter)

 Define the tanh domain transform as: T j   ( L j )  tanh( L j / 2)

 The syndrome of a parity check can be expressed as: si  r
H ij 1
j

 Define the soft syndrome as: Si   T   ( L )

H ij 1
j
H ij 1
j

nk
 Define the cost function as: J ( H , T )   Si
i 1

The decoding problem is relaxed as minimizing J using gradient

descent with the initial value T observed from the channel

J is also a function of H. It is adapted such that unreliable bits are

separated in order to avoid getting stuck at zero gradient points: H   (H ( 0) , T)
Two Stage Optimization Procedure

Proposed algorithm is a generalization of the iterative decoding scheme

proposed by Lucas et al. (1998), two-stage optimization procedure:

H (t )   (H (0) , T(t ) )

nk
T ( t 1)
m   ( (T )   ( 1 (
1 (t )
m  m )))
T (t )

i 1 j{ j | H ij 1, j  m}

 The damping coefficient serves to control the convergent dynamics

Avoid Zero Gradient Point

Adaptive scheme
changes the gradient
and prevents it
getting stuck at zero
gradient points

Zero gradient
point
Variations of the Generic Algorithm

 Connect unreliable bits as deg-2

 Incorporate this algorithm with hard decision decoder

 Adapting the parity check matrix at symbol level

 Exchange bits in reliable and unreliable part. Run the decoder multiple times

 Reduced complexity partial updating scheme

Simulation Results
AWGN Channels
AWGN Channels (cont’d)

Asymptotic performance
is consistent with the ML
upper-bound.
AWGN Channels (cont’d)
AWGN Channels (cont’d)
Interleaved Slow Fading Channel
Fully Interleaved Slow Fading Channels
Fully Interleaved Slow Fading Channels (cont.)
Turbo Equalization Systems
Embed the Proposed Algorithm in the Turbo Equalization System

source interleaving

RS Encoder Π PR Encoder

+
interleaving
AWGN
extrinsic a priori Σ
+
Σ Π +

-
sink de-interleaving
BCJR
RS
Decoder a priori
Π 1 extrinsic
Equalizer
hard decision

RS Coded Turbo Equalization System

Turbo Equalization over EPR4 Channels
Turbo Equalization over EPR4 Channels
RS Coded Modulation
RS Coded Modulation over Fast Rayleigh Fading Channels
Applications and Future Works
Potential Problems in Applications

 “In theory, there is no difference between theory and practice.

But, in practice, there is…” (Jan L.A. van de Snepscheut)

 General Problems:
Coding gain may shrink down in practical systems
Concatenated with CC: difficult to generate the soft information

Performance in the practical SNRs should be analyzed

 Respective problems for various decoding schemes:

Reliability assisted HDD: Gain is marginal in practical SNRs
Algebraic soft decoding: performance is limited by the algebraic nature

Reliability based decoding: huge memory, not scalable with SNR

Sub-code decomposition: only possible for very short codes

Iterative decoding: adapting H at each iteration is a huge cost

b
A Case Study (System Setups)

 Forward Error Control of a Digital Television Transmission Standard:

Modulation format: 64 or 16 QAM modulation (semi-set partitioning mapping)

Inner code: convolutional code rate=2/3 or 8/9

Bit-interleaved coded modulation (BICM)

Iterative demodulation and decoding (BICM-ID)

The decoded bytes from inner decoder are interleaved and fed to outer decoder

Outer code: RS(208,188) using hard decision decoding

 Will soft decoding algorithm significantly improve the overall performance?

A Case Study (Simulation Results)
Future Works

How to incorporate the proposed ADP with other soft decoding

schemes?

Taking advantage of the inherent structure of RS codes at bit level

More powerful decoding tool, e.g., trellis

Extend the idea of adaptive algorithms to demodulation and equalization

Apply the ADP algorithm to quantization or to solve K-SAT problems

Thank you!