## Modern coding theory: the statistical mechanics and computer science point of view (2007)

Venue: | SUMMER SCHOOL ON COMPLEX SYSTEMS, LES HOUCHES |

Citations: | 9 - 0 self |

### BibTeX

@INPROCEEDINGS{Montanari07moderncoding,

author = {Andrea Montanari and Rüdiger Urbanke},

title = {Modern coding theory: the statistical mechanics and computer science point of view },

booktitle = {SUMMER SCHOOL ON COMPLEX SYSTEMS, LES HOUCHES},

year = {2007},

pages = {704--2857},

publisher = {Online]. Available}

}

### OpenURL

### Abstract

These are the notes for a set of lectures delivered by the two authors at the Les Houches Summer School on ‘Complex Systems’ in July 2006. They provide an introduction to the basic concepts in modern (probabilistic) coding theory, highlighting connections with statistical mechanics. We also stress common concepts with other disciplines dealing with similar problems that can be generically referred to as ‘large graphical models’. While most of the lectures are devoted to the classical channel coding problem over simple memoryless channels, we present a discussion of more complex channel models. We conclude with an overview of the main open challenges in the field.

### Citations

10958 |
Computers and Intractability: A Guide to the Theory of NP-Completeness
- Garey, Johnson
- 1979
(Show Context)
Citation Context ...i ν (t) j→a (xj). (5.3) ν (t) b→i (xi). (5.4) The basic theoretical question is of course to establish a relation, if any between µi( · ) and νi( · ). As an example, we shall consider satisfiability =-=[23]-=-. Given N Boolean variables xi, i ∈ {1, . . .,N}, xi ∈ {True, False}, a formula is the logical expression obtained by taking the AND of M clauses. Each clause is the logical OR of a subset of the vari... |

8595 |
Elements of Information Theory
- Cover, Thomas
- 1991
(Show Context)
Citation Context ...ooks Information, Physics and Computation [36] and Modern Coding Theory [50]. Standard references on coding theory are [6, 9, 26] and very readable introductions to information theory can be found in =-=[12, 20]-=-. Other useful reference sources are the book by Nishimori [44] as well as the book by MacKay [29]. 2 Background: The Channel Coding Problem The central problem of communications is how to transmit in... |

6080 |
A mathematical theory of communication
- Shannon
- 1948
(Show Context)
Citation Context ...cience. The underlying reason for this convergence is the importance of probabilistic models and/or probabilistic techniques in each of these domains. This has long been obvious in information theory =-=[53]-=-, statistical mechanics [10], and statistical inference [45]. In the last few years it has also become apparent in coding theory and theoretical computer science. In the first case, the invention of T... |

1870 | Randomized Algorithms
- Motwani, Raghavan
- 1995
(Show Context)
Citation Context ...dom constructions for coding information in robust/compact ways [50]. In the second case (theoretical computer science) the relevance of randomized algorithms has steadily increased (see for instance =-=[41]-=-), thus motivating deep theoretical developments. A particularly important example is provided by the Monte Carlo Markov Chain method for counting and sampling random structures. Given this common pro... |

1306 | Near Shannon limit errorcorrecting coding and decoding: Turbo codes
- Berrou, Glavieux, et al.
- 1993
(Show Context)
Citation Context ...al mechanics [10], and statistical inference [45]. In the last few years it has also become apparent in coding theory and theoretical computer science. In the first case, the invention of Turbo codes =-=[7]-=- and the re-invention of Low-Density Parity-Check (LDPC) codes [30, 28] has motivated the use of random constructions for coding information in robust/compact ways [50]. In the second case (theoretica... |

1165 |
Information Theory, Inference, and Learning Algorithms
- MacKay
- 2003
(Show Context)
Citation Context ...oding theory are [6, 9, 26] and very readable introductions to information theory can be found in [12, 20]. Other useful reference sources are the book by Nishimori [44] as well as the book by MacKay =-=[29]-=-. 2 Background: The Channel Coding Problem The central problem of communications is how to transmit information reliably through a noisy (and thus unreliable) communication channel. Coding theory aims... |

893 | Low-density parity-check codes
- Gallager
- 1963
(Show Context)
Citation Context ... least one sparse parity-check matrix. Rather than considering the most general case let us limit ourselves to a particularly simple family of LDPC ensembles, originally introduced by Robert Gallager =-=[19]-=-. We call them ‘regular’ ensembles. An element in this family is characterized by the blocklength N and two integer numbers k and l, with k > l. We shall therefore refer to it as the (k, l) regular en... |

514 | Good error-correcting codes based on very sparse matrices
- MacKay
- 1999
(Show Context)
Citation Context ... years it has also become apparent in coding theory and theoretical computer science. In the first case, the invention of Turbo codes [7] and the re-invention of Low-Density Parity-Check (LDPC) codes =-=[30, 28]-=- has motivated the use of random constructions for coding information in robust/compact ways [50]. In the second case (theoretical computer science) the relevance of randomized algorithms has steadily... |

419 |
Theory and practice of error control codes
- Blahut
- 1983
(Show Context)
Citation Context ...heory. A considerably more in-depth discussion can be found in the two upcoming books Information, Physics and Computation [36] and Modern Coding Theory [50]. Standard references on coding theory are =-=[6, 9, 26]-=- and very readable introductions to information theory can be found in [12, 20]. Other useful reference sources are the book by Nishimori [44] as well as the book by MacKay [29]. 2 Background: The Cha... |

413 | Constructing free energy approximations and generalized belief propagation algorithms
- YEDIDIA, FREEMAN, et al.
(Show Context)
Citation Context ...tion to statistical models on non-tree graphs. This is often referred to as the Bethe-Peierls approximation [8]. The Bethe approximation presents several advantages with respect to ‘naive-mean field’ =-=[61]-=- (that amounts to writing ‘self-consistency’ equations for expectations over single degrees of freedom). It retains correlations among degrees of freedom that interact directly, and is exact on some n... |

374 | Structuring in Belief Networks
- Pearl, “Fusion
- 1986
(Show Context)
Citation Context ...portance of probabilistic models and/or probabilistic techniques in each of these domains. This has long been obvious in information theory [53], statistical mechanics [10], and statistical inference =-=[45]-=-. In the last few years it has also become apparent in coding theory and theoretical computer science. In the first case, the invention of Turbo codes [7] and the re-invention of Low-Density Parity-Ch... |

280 | Expander codes
- Sipser, Spielman
- 1996
(Show Context)
Citation Context ...n yields pc ≈ 0.1091. Bit-flipping is significantly sub-optimal, but it is still surprisingly good, given the extreme simplicity of the algorithm. These numerical findings can be confirmed rigorously =-=[55]-=-. Theorem 4.1. Consider a regular (l, k) LDPC ensemble and let C be chosen uniformly at random from the ensemble. If l ≥ 5 then there exists ε > 0 such that, with high probability, Bit-flipping is abl... |

228 | Practical Loss-Resilient Codes
- Luby, Mitzenmacher, et al.
- 1997
(Show Context)
Citation Context ...eneric row/column weight distribution (the weight being the number of non-zero elements), cf. Fig. 6 for an illustration. Such ensembles are usually referred to as ‘irregular’, and were introduced in =-=[27]-=-. 11degree 2 variables degree 3 variables { }} { { }} { degree dmax variables { }} { permutation π } {{ } } {{ } degree 2 checksdegree 3 checks } {{ } degree dmax checks Figure 6: Factor graph of an ... |

219 |
Iterative correction of intersymbol interference: turbo-equalization
- Douillard, Jezequel, et al.
- 1995
(Show Context)
Citation Context ... the general situation: as long as the channel memory can be described by a Markov chain the factor graph approach applies and we can use message-passing schemes to construct efficient coding schemes =-=[14, 4, 21, 22, 25, 43]-=-. 6.3 Asymmetric Channels - The Z Channel Let us now consider the second generalization, namely the case of non-symmetric channels. Consider the channel depicted on the right of Fig. 1. For obvious re... |

169 | The capacity of low-density parity check codes under message-passing decoding
- Richardson, Urbanke
(Show Context)
Citation Context ... bold continuous lines to the limit t → ∞. In the inset we plot the expected conditional entropy E H(Xi|ν (t) i ). efficient alternatives in the coding context (mainly based on Fourier transform, see =-=[49, 48]-=-), we shall describe population dynamics because it is easily programmed. Let us describe the algorithm within the setting of a general message passing decoder, cf. Eq. (4.17). Given an integer N ≫ 1,... |

121 |
Codes on graphs: Normal Realizations
- Forney
- 2001
(Show Context)
Citation Context ...egree two and therefore connect exactly two factors). Such a graphical representation is also known as normal graph or as Forney-style factor graph (FSFG), in honor of Dave Forney who introduced them =-=[15]-=-. Let us now look at a concrete example. x1 x2 x3 xn σ0 σ1 σ2 σ3 σn−1 σn p(σ0) p(x2,y2,σ2 | σ1) p(x1,y1,σ1 | σ0) p(x3,y3,σ3 | σ2) Figure 17: The FSFG corresponding to (6.2). p(xn,yn,σn | σn−1) Example... |

79 | Good codes based on very sparse matrices
- Mackay, Neal
- 1995
(Show Context)
Citation Context ... years it has also become apparent in coding theory and theoretical computer science. In the first case, the invention of Turbo codes [7] and the re-invention of Low-Density Parity-Check (LDPC) codes =-=[30, 28]-=- has motivated the use of random constructions for coding information in robust/compact ways [50]. In the second case (theoretical computer science) the relevance of randomized algorithms has steadily... |

75 | Loopy belief propagation and Gibbs measures
- Tatikonda, Jordan
(Show Context)
Citation Context ... which is called survey propagation [11, 33, 11]. 5.1 Proving Correctness through Correlation Decay A simple trick to bound the error incurred by BP consists in using the correlation decay properties =-=[1, 57]-=- of the measure µ( · ). Let i ∈ {1, . . ., N} be a variable index and denote by xt the set of variables sitting on nodes at distance t from i. Further, denote by x≥t the set of variables whose distanc... |

65 |
Probabilistic analysis of the Davis–Putnam procedure for solving the satisfiability problem
- Franco, Paull
(Show Context)
Citation Context ... ,x′ t |µ(xi|xt) − µ(xi|x′ t )| ≤ exp(−κt) for some positive κ. This implies |µi(xi) − ν (t) i (xi)| ≤ e −κg/2 . (5.9) As an example of such error estimates, we shall consider random k-satisfiability =-=[16]-=-. This is a standard model to generate ‘synthetic’ satisfiability formulae. It amounts to picking a formula uniformly at random among all the ones including N variables and M = Nα k-clauses (a k-claus... |

58 |
A theory of cooperative phenomena
- Kikuchi
- 1951
(Show Context)
Citation Context ...y described in Section 5. We should finally mention that several improvements over Bethe approximation have been developed within statistical physics. Among them, Kikuchi’s cluster variational method =-=[24]-=- is worth mentioning since it motivated the development of a ‘generalized belief propagation’ algorithm, which spurred a lot of interest within the artificial intelligence community [61]. 4.4 Density ... |

47 | A new look at survey propagation and its generalizations
- Maneva, Mossel, et al.
(Show Context)
Citation Context ...ormly. These are well known #-P complete problems. 14 The currently best algorithms for solving random instances for the K-SAT problem are based on a variant of BP, which is called survey propagation =-=[11, 33, 11]-=-. 5.1 Proving Correctness through Correlation Decay A simple trick to bound the error incurred by BP consists in using the correlation decay properties [1, 57] of the measure µ( · ). Let i ∈ {1, . . .... |

37 | Iterative equalization and decoding in mobile communications systems
- Bauch, Khorram, et al.
- 1997
(Show Context)
Citation Context ... the general situation: as long as the channel memory can be described by a Markov chain the factor graph approach applies and we can use message-passing schemes to construct efficient coding schemes =-=[14, 4, 21, 22, 25, 43]-=-. 6.3 Asymmetric Channels - The Z Channel Let us now consider the second generalization, namely the case of non-symmetric channels. Consider the channel depicted on the right of Fig. 1. For obvious re... |

35 |
Spin-glass models as error-correcting codes
- Sourlas
- 1989
(Show Context)
Citation Context ...se, are these close to each other or very different? And what is their relationship with the transmitted codeword? The connection to statistical mechanics emerges in the study of the decoding problem =-=[56, 51]-=-. To make it completely transparent we rewrite the distribution µ( · ) in Boltzmann form µC,y(x) = EC,y(x) = 1 Z(C, y) e−EC,y(x) , (2.9) { ∑N − i=1 log Q(yi|xi), if x ∈ C , (2.10) +∞, otherwise. The w... |

28 | Low-density parity-check codes for Gilbert-Elliot and Markov-modulated channels
- Eckford
- 2004
(Show Context)
Citation Context ... the general situation: as long as the channel memory can be described by a Markov chain the factor graph approach applies and we can use message-passing schemes to construct efficient coding schemes =-=[14, 4, 21, 22, 25, 43]-=-. 6.3 Asymmetric Channels - The Z Channel Let us now consider the second generalization, namely the case of non-symmetric channels. Consider the channel depicted on the right of Fig. 1. For obvious re... |

28 |
The Bethe lattice Spin glass revisited
- Mézard, Parisi
- 2001
(Show Context)
Citation Context ... cavity equations. With respect to Bethe-Peierls, the cavity approach includes a hierarchy of (‘replica symmetry breaking’) refinements of such equations that aim at capturing long range correlations =-=[37]-=-. This will briefly described in Section 5. We should finally mention that several improvements over Bethe approximation have been developed within statistical physics. Among them, Kikuchi’s cluster v... |

24 |
Performance Analysis of Linear Codes under Maximum-Likelihood Decoding: A Tutorial,” Foundations and Trends in
- Sason, Shamai
- 2006
(Show Context)
Citation Context ...ing a random code from the ensemble, provided N is large enough. By the optimality of MAP decoding, pd ≤ pc. In coding theory some techniques have been developed to prove upper and lower bounds on pc =-=[19, 52]-=-. In particular it is easy to find ensembles for which there exist a gap between the two thresholds (namely pd < pc strictly). Consider for instance (k, l) regular ensembles with a fixed ratio l/k = 1... |

22 | Finitelength scaling for iteratively decoded LDPC ensembles
- Amraoui, Montanari, et al.
- 2009
(Show Context)
Citation Context ...The typical convergence of the performance to the asymptotic limit is shown in Fig. 21. The points correspond to simulation results whereas the solid curves correspond to a general scaling conjecture =-=[2]-=-. Roughly speaking, this scaling conjecture states that around the threshold the error probability behaves as follows: Let the channel be parameterized by ǫ with increasing ǫ indicating a worsening of... |

22 | Are turbo-like codes effective on nonstandard channels
- McEliece
- 2001
(Show Context)
Citation Context ...es into a non-uniform input distribution at the channel input in order to bring the mutual information closer to capacity. For a detailed discussion on coding for the Z-channel we refer the reader to =-=[34, 59, 5]-=-. 7 Open Problems Let us close by reviewing some of the most important open challenges in the channel coding problem. 7.1 Order of Limits Density evolution computes the limit lim t→∞ lim N→∞ E[P(N,t) ... |

20 | Combining hidden Markov source models and parallel concatenated codes
- Garcia-Frias, Villasenor
- 1997
(Show Context)
Citation Context |

20 | Density evolution for asymmetric memoryless channels
- Wang, Kulkarni, et al.
- 2005
(Show Context)
Citation Context ...es into a non-uniform input distribution at the channel input in order to bring the mutual information closer to capacity. For a detailed discussion on coding for the Z-channel we refer the reader to =-=[34, 59, 5]-=-. 7 Open Problems Let us close by reviewing some of the most important open challenges in the channel coding problem. 7.1 Order of Limits Density evolution computes the limit lim t→∞ lim N→∞ E[P(N,t) ... |

16 |
The objective method
- Aldous, Steele
- 2002
(Show Context)
Citation Context ...mmetric channel BSC(p) is defined by letting A = {0,1} and Q(0|0) = Q(1|1) = 1−p; the normalization then enforces Q(1|0) = Q(0|1) = p. In words, the channel “flips” the input bit with probability p ∈ =-=[0, 1]-=-. Since flips are introduced for each bit independently we 1 It is worth mentioning that an alternative approach would be to consider the noise as ‘adversarial’ (or worst case) under some constraint o... |

16 | The glassy phase of gallager codes
- Montanari
- 2001
(Show Context)
Citation Context ... corresponding derivations. Finally, it would be interesting to have a sharp estimate for pc. It turns out that a sharp characterization of pc can be obtained through statistical mechanics techniques =-=[42, 58, 38]-=-. The statistical mechanics result has been proved to be a rigorous upper bound for general code ensembles, and it is conjectured to be tight [39, 35]. The starting point is to consider the conditiona... |

15 |
Vorlesungen über Gastheorie
- Boltzmann
- 1896
(Show Context)
Citation Context ...n for this convergence is the importance of probabilistic models and/or probabilistic techniques in each of these domains. This has long been obvious in information theory [53], statistical mechanics =-=[10]-=-, and statistical inference [45]. In the last few years it has also become apparent in coding theory and theoretical computer science. In the first case, the invention of Turbo codes [7] and the re-in... |

14 | The dynamic phase transition for decoding algorithms
- Franz, Leone, et al.
- 2002
(Show Context)
Citation Context ...uctural properties of µ( · ) that are commonly regarded as incompatible with replica symmetry breaking. Replica symmetry breaking is instead necessary to describe the structure of ‘metastable states’ =-=[17]-=-. This can be loosely described as very deep local minima in the energy landscape introduced in Section 4.1. Here ‘very deep’ means that Θ(N) bit flips are necessary to lower the energy (number of uns... |

14 |
Finite temperature error–correcting codes,” Phys
- Ruján
- 1993
(Show Context)
Citation Context ...se, are these close to each other or very different? And what is their relationship with the transmitted codeword? The connection to statistical mechanics emerges in the study of the decoding problem =-=[56, 51]-=-. To make it completely transparent we rewrite the distribution µ( · ) in Boltzmann form µC,y(x) = EC,y(x) = 1 Z(C, y) e−EC,y(x) , (2.9) { ∑N − i=1 log Q(yi|xi), if x ∈ C , (2.10) +∞, otherwise. The w... |

12 | Statistical physics of regular low-density parity-check error-correcting codes,” Phys
- Murayama, Kabashima, et al.
- 2000
(Show Context)
Citation Context ... corresponding derivations. Finally, it would be interesting to have a sharp estimate for pc. It turns out that a sharp characterization of pc can be obtained through statistical mechanics techniques =-=[42, 58, 38]-=-. The statistical mechanics result has been proved to be a rigorous upper bound for general code ensembles, and it is conjectured to be tight [39, 35]. The starting point is to consider the conditiona... |

11 |
Algebraic Coding Theory, Aegean
- Berlekamp
- 1984
(Show Context)
Citation Context ...heory. A considerably more in-depth discussion can be found in the two upcoming books Information, Physics and Computation [36] and Modern Coding Theory [50]. Standard references on coding theory are =-=[6, 9, 26]-=- and very readable introductions to information theory can be found in [12, 20]. Other useful reference sources are the book by Nishimori [44] as well as the book by MacKay [29]. 2 Background: The Cha... |

10 |
Statistical theory of superlattices,” Proc
- Bethe
- 1935
(Show Context)
Citation Context ... solve exactly models on trees. However they have been often applied as mean-field approximation to statistical models on non-tree graphs. This is often referred to as the Bethe-Peierls approximation =-=[8]-=-. The Bethe approximation presents several advantages with respect to ‘naive-mean field’ [61] (that amounts to writing ‘self-consistency’ equations for expectations over single degrees of freedom). It... |

10 |
Fast density evolution
- Jin, Richardson
- 2004
(Show Context)
Citation Context ... bold continuous lines to the limit t → ∞. In the inset we plot the expected conditional entropy E H(Xi|ν (t) i ). efficient alternatives in the coding context (mainly based on Fourier transform, see =-=[49, 48]-=-), we shall describe population dynamics because it is easily programmed. Let us describe the algorithm within the setting of a general message passing decoder, cf. Eq. (4.17). Given an integer N ≫ 1,... |

9 | Recipes for metastable states in spin glasses
- Franz, Parisi
- 1995
(Show Context)
Citation Context ...the correct exponential rate. This claim has been supported rigorously by the results of [47, 3, 31]. Let us finally comment on the relation between distance enumerator and the Franz-Parisi potential =-=[18]-=-, introduced in the study of glassy systems. In this context the potential is used to probe the structure of the Boltzmann measure. One considers a system with energy function E(x), a reference config... |

8 |
Jr., “Two results on binary-input discrete memoryless channels
- Majani, Rumsey
- 1991
(Show Context)
Citation Context ...n is at least a fraction 1 2e ln(2) ≈ 0.924 of capacity over the entire range of p (with equality when p approaches one). Even more fortunate, from this perspective the Z channel is the extremal case =-=[32, 54]-=-: the information rate of any binary-input memoryless channel when the input distribution is the uniform one is at least a fraction 1 2e ln(2) of its capacity. From the above discussion we conclude th... |

7 |
Iterative decoding of LDPC codes over arbitrary discrete-memoryless channels
- Bennatan, Burshtein
- 2003
(Show Context)
Citation Context ...es into a non-uniform input distribution at the channel input in order to bring the mutual information closer to capacity. For a detailed discussion on coding for the Z-channel we refer the reader to =-=[34, 59, 5]-=-. 7 Open Problems Let us close by reviewing some of the most important open challenges in the channel coding problem. 7.1 Order of Limits Density evolution computes the limit lim t→∞ lim N→∞ E[P(N,t) ... |

7 | Finite size scaling for the core of large random hypergraphs, Annals of Applied Probability 18
- Dembo, Montanari
- 2008
(Show Context)
Citation Context ...is smaller than x), and α is a constant which depends on the channel as well as on the channel. For the BEC this scaling law has been shown to be correct [2]. In fact, even a refined version is known =-=[13]-=-: define z = √ 2 − n(ǫ − ǫd + βn 3) where β is a constant depending on the ensemble. Then for z fixed and n tending to infinity we have PB(n, ǫ) = Q (z/α) ( 1 + O ( n −1/3)) . For general channels on ... |

7 | Counting good truth assignments of random k-SAT formulae
- Montanari, Shah
- 2007
(Show Context)
Citation Context ... − µ(xi|x t ′ t )| ≤ e−κt (5.10) can be proved if the clause density α stays below a threshold αu(k) that is estimated to behave as 2 log k αu(k) = k [1 + ok(1)]. While we refer to the original paper =-=[40]-=- for the details of the proof we limit ourselves to noticing that the left hand side of Eq. (5.10) can be estimated efficiently using a density evolution procedure. This allows to estimate the thresho... |

6 |
Self-consistent theory of localization. II. localization near the band edges
- Abou-Chacra, Thouless
- 1974
(Show Context)
Citation Context ...es of some fixed size. Within statistical physics this is sometimes called the population dynamics algorithm (and made its first appearance in the study of the localization transition on Cayley trees =-=[46]-=-). Although there exist more 11 The suspicious reader will notice that this is not exactly a particular case of the previous statement, because f(x) = I(x < 0) + 1 I(x = 0) is not a continuous functio... |

5 |
bounds on the spectrum and error rate of LDPC code ensembles
- Barak, Burshtein, et al.
- 2005
(Show Context)
Citation Context ...e other hand, for regular ensembles as the ones considered here, φq(ω) = φ(ω): the annealed calculation yields the correct exponential rate. This claim has been supported rigorously by the results of =-=[47, 3, 31]-=-. Let us finally comment on the relation between distance enumerator and the Franz-Parisi potential [18], introduced in the study of glassy systems. In this context the potential is used to probe the ... |

5 |
The generalized area theorem and some of its consequences,” submitted to
- Measson, Montanari, et al.
- 2005
(Show Context)
Citation Context ...ined through statistical mechanics techniques [42, 58, 38]. The statistical mechanics result has been proved to be a rigorous upper bound for general code ensembles, and it is conjectured to be tight =-=[39, 35]-=-. The starting point is to consider the conditional entropy of the channel input x given the output y, HN(X|Y ). As shown in Eq. (2.16) this is given by the expectation of the log partition function a... |

4 |
the asymptotic weight and stopping setdistribution ofregular LDPCensembles
- Rathi
- 2006
(Show Context)
Citation Context ...e other hand, for regular ensembles as the ones considered here, φq(ω) = φ(ω): the annealed calculation yields the correct exponential rate. This claim has been supported rigorously by the results of =-=[47, 3, 31]-=-. Let us finally comment on the relation between distance enumerator and the Franz-Parisi potential [18], introduced in the study of glassy systems. In this context the potential is used to probe the ... |

3 |
Correlation inequalities: a useful tool in the theory of LDPC codes
- Macris
- 2005
(Show Context)
Citation Context ...e other hand, for regular ensembles as the ones considered here, φq(ω) = φ(ω): the annealed calculation yields the correct exponential rate. This claim has been supported rigorously by the results of =-=[47, 3, 31]-=-. Let us finally comment on the relation between distance enumerator and the Franz-Parisi potential [18], introduced in the study of glassy systems. In this context the potential is used to probe the ... |

3 |
bounds for LDPC and LDGM codes under MAP decoding
- “Tight
- 2005
(Show Context)
Citation Context ...ined through statistical mechanics techniques [42, 58, 38]. The statistical mechanics result has been proved to be a rigorous upper bound for general code ensembles, and it is conjectured to be tight =-=[39, 35]-=-. The starting point is to consider the conditional entropy of the channel input x given the output y, HN(X|Y ). As shown in Eq. (2.16) this is given by the expectation of the log partition function a... |