ITW 1999.

Metsovo, Greece, June 27 - July 1

Factor Graphs and Iterative Algorithms

Ralf Koetter
Coordinated Science Laboratory, University of Illinois, 1308 W. Main Street, Urbana, IL 61801, U.S.A. koetterQuiuc.edu

Abstract - We review the notion of factor graphs and give an overview of recent developments in the field.

I. INTRODUCTION
Turbo decoding and other iterative decoding techniques have in the last few years proved t o be a milestone development in coding theory. The unsurpassed performance-complexity tradeoff of iteratively decodable codes has led t o an explosion of work in this area. The goals of this research were focussed two classical problems in coding theory: analysis of decoding algorithms, and construction of codes. Factor graphs [l] provide a framework in which iterative algorithms and iteratively decodable codes are easily described. They are obtained as a slight modification of the graphs, which were introduced by Wiberg, Loeliger and Koetter in [2]. The notion of factor graphs, defined in Section 11, is related to other graphical models, such as Bayesian networks and Markov Random Fields, and allows an easy integration of the decoding task and related problems in a communication system, such as channel estimation and equalization. Recently, a number of research topics involving factor graphs have developed. These include the construction and representation of codes on graphs and an analysis of iterative decoding algorithm stemming from the theory of dynamical systems.

The decoding of codes on factor graphs uses the sumproduct algorithm in its various forms. A generalized sumproduct algorithm encompasses a variety of different algorithms under appropriate definitions of addition and multiplication, without changing the essence of the algorithm. The algorithm proceeds by sending messages concerning the value of a variable along the edges of the factor graph. The messages represent the belief that the node sending the message has about the value of the variable in question. For a detailed treatment of the sum-product algorithm, we refer the reader t o [I].

An interesting approach t o the analysis of iterative decoding investigates the behavior of a decoding algorithm in infinite, cycle-free, factor graphs. The approach was first used by Gallager[3]. It was recently refined by Luby, Mitzenmacher, Shokrollahi, and Spielman .[4] and Urbanke and Richardson[5]. A message being passed in the factor graph is a random variable that depends on the received channel values. In the simplest setup, if the underlying factor graph is cycle-free and regular (cf. [4]), messages being sent t o a node are independent and identically distributed. It is possible to investigate the evolution of probability densities of messages. The assumption that the graph is cycle-free is justified by the fact that graphs with arbitrarily long cycles can be constructed. Hence the assumption can be satisfied for an arbitrary number of decoding rounds. Figure 2 shows the bit11. FACTOR GRAPHS error rate that is achievable with a [2,3] low-density-parityLet x = ( ~ 0 ~ x 1. , ,.. } be a vector of variables. A factor check-code (LDPC) [3] for codelength and the number of de. x, graph visualizes the factorization of a global function f ( x ) . coding iterations approach infinity. Let f (x)factor as f (x) = f,(x(2)), where x(*)is the set of variables of the function f,. A factor graph for f is defined as the bipartite graph with two vertex classes Vf and V, of size m and n such that the ith node in Vf is incident to the j t h node in V, if and only if f, is a function of 2,. The variables may be from different alphabets as long as the functions fi BER are properly d e h e d . 0 1 2 SNR/[db] Figure 2. Asymptotic performance of a [2,3] LDPC on an - AWGN with -a message passing decoding-algorithm

111. THEDYNAMICAL SYSTEMS APPROACH

n= :,

I REFERENCES

Figure 1. A factor graph for a [6,3,3] binary code @. Function nodes are drawn as small filled circles and variable nodes are drawn as larger unfilled circles.The globa! function f ( x ) equals one if x is a word in C and equals zero otherwise. The local functions fi (x()) equal one if the length three vectors x(;)have even weight and equals zero otherwise.
The definition of multiplication and addition may be taken rather generally. For the precise algebraic properties that multiplication and addition have to satisfy, see [l].

F. Kschischang, B. Frey, and H.-A.Loeliger, Factor Graphs and the Sum-Product Algorithm, submitted to IEEE Dans. Inform. Theory, July 1998 N. Wiberg, H.-A.Loeliger and R. Koetter, Codes and iterative decoding on general graphs, Euro. T h n s . Telecornmun., vol. 6, pp. 513-526, September 1995. R.G. Gallager, Low-Density Parity-Check Codes, M.I.T. Press,
1963

M. Luby, M. Mitzenmacher, A. Shokrollahi, and D.Spielman, Analysis of Low Density Codes and Improved Designs Using Irregular Graphs 30th ACM STOC, Dallas, May 1998. T. Richardson and R Urbanke, The Capacity of LowDensity Parity Check Codes under Message-Passing Decoding, http://crn. bell-labs .corn/who/tjr/pub. htrnl, preprint, 1998

28