## Design of capacity-approaching irregular low-density parity-check codes (2001)

Venue: | IEEE TRANS. INFORM. THEORY |

Citations: | 457 - 7 self |

### BibTeX

@ARTICLE{Richardson01designof,

author = {Thomas J. Richardson and M. Amin Shokrollahi and Rüdiger L. Urbanke},

title = {Design of capacity-approaching irregular low-density parity-check codes},

journal = {IEEE TRANS. INFORM. THEORY},

year = {2001},

volume = {47},

pages = {619--637}

}

### Years of Citing Articles

### OpenURL

### Abstract

We design low-density parity-check (LDPC) codes that perform at rates extremely close to the Shannon capacity. The codes are built from highly irregular bipartite graphs with carefully chosen degree patterns on both sides. Our theoretical analysis of the codes is based on [1]. Assuming that the underlying communication channel is symmetric, we prove that the probability densities at the message nodes of the graph possess a certain symmetry. Using this symmetry property we then show that, under the assumption of no cycles, the message densities always converge as the number of iterations tends to infinity. Furthermore, we prove a stability condition which implies an upper bound on the fraction of errors that a belief-propagation decoder can correct when applied to a code induced from a bipartite graph with a given degree distribution. Our codes are found by optimizing the degree structure of the underlying graphs. We develop several strategies to perform this optimization. We also present some simulation results for the codes found which show that the performance of the codes is very close to the asymptotic theoretical bounds.

### Citations

3180 | Probability and Measure - Billingsley - 1995 |

1518 |
Information Theory and Reliable Communication
- Gallager
- 1968
(Show Context)
Citation Context ...nating from all variable nodes, is equal to 1 It is reassuring to note that linear binary codes are known to be capable of achieving capacity on binary-input memoryless output-symmetric channels, see =-=[9]-=-. (1) In the same manner, assuming that the code has check nodes, can also be expressed as Equating these two expressions for , we conclude that Generically, assuming that all these check equations ar... |

1370 | Near Shannon limit error-correcting coding and decoding
- Berrou, Glavieux, et al.
- 1993
(Show Context)
Citation Context ...nce of the best irregular LDPC ensemble we found in our search and with the performance of an instance of the standard parallel concatenated ensemble introduced by Berrou, Glavieux, and Thitimajshima =-=[12]-=-. All three codes have rate one-half and their performance under belief-propagation decoding over the BIAWGNC is shown for a code word length of . Also shown is the Shannon limit and the threshold val... |

963 | Low-Density Parity-Check Codes
- Gallager
- 1963
(Show Context)
Citation Context ...for the best degree distribution pair with some a priori bound on the size of the degrees. 6 In the case of maximum-likelihood decoding this was answered in the affirmative by Gallager and McKay, see =-=[13]-=-, [14]. 7 We conjecture that a similar statement (and proof) can be given for continuous channels. 8 In fact, a similar theorem holds also for the erasure channel [15, Theorem 1], and yet, there are c... |

552 | Good error-correcting codes based on very sparse matrices
- MacKay
- 1999
(Show Context)
Citation Context ...e best degree distribution pair with some a priori bound on the size of the degrees. 6 In the case of maximum-likelihood decoding this was answered in the affirmative by Gallager and McKay, see [13], =-=[14]-=-. 7 We conjecture that a similar statement (and proof) can be given for continuous channels. 8 In fact, a similar theorem holds also for the erasure channel [15, Theorem 1], and yet, there are capacit... |

469 |
A recursive approach to low complexity codes
- Tanner
- 1981
(Show Context)
Citation Context ... graphs, see [2], [24], [3]. An alternative solution for practical purposes, which does not require cascades, is presented in [4]. Let us recall some basic notation. As originally suggested by Tanner =-=[5]-=-, LDPC codes are well represented by bipartite graphs in which one set of nodes, the variable nodes, corresponds to elements of the codeword and the other set of nodes, the check nodes, corresponds to... |

386 | The capacity of low-density parity-check codes under message-passing decoding
- Richardson, Urbanke
(Show Context)
Citation Context ...remely close to the Shannon capacity. The codes are built from highly irregular bipartite graphs with carefully chosen degree patterns on both sides. Our theoretical analysis of the codes is based on =-=[1]-=-. Assuming that the underlying communication channel is symmetric, we prove that the probability densities at the message nodes of the graph possess a certain symmetry. Using this symmetry property we... |

237 | Practical loss-resilient codes
- Luby, Mitzenmacher, et al.
- 2001
(Show Context)
Citation Context ... which connects the distributions of messages passed from variable nodes to check nodes at two consecutive iterations of belief propagation. In the case of the BEC, this task has been accomplished in =-=[2]-=-, [24], [10] where it was shown that the expected fraction of erasure messages which are passed in the th iteration, call it , evolves as For general binary-input memoryless output-symmetric channels,... |

231 | On the design of low-density parity-check codes within 0.0045 dB of the Shannon limit - Chung, Forney, et al. - 2001 |

160 |
Differential evolution-a simple and efficient adaptive scheme for global optimization over
- Storn, Price
- 1995
(Show Context)
Citation Context ...int satisfaction problems with continuous space parameters. Many general algorithms for solving such problems have been developed. We experimented with an algorithm called Differential Evolution (DE) =-=[23]-=- that has already been successfully applied to the design of good erasure codes [11]. DE is a robust optimizer for multivariate functions. We will not describe the details here, suffice it to say that... |

134 |
Urbanke, “Efficient encoding of low-density parity-check codes
- Richardson, L
- 2001
(Show Context)
Citation Context ...odify the construction of codes from bipartite graphs to a cascade of such graphs, see [2], [24], [3]. An alternative solution for practical purposes, which does not require cascades, is presented in =-=[4]-=-. Let us recall some basic notation. As originally suggested by Tanner [5], LDPC codes are well represented by bipartite graphs in which one set of nodes, the variable nodes, corresponds to elements o... |

133 | Large deviations for performance analysis
- Shwartz, Weiss
- 1995
(Show Context)
Citation Context ...ity will evolve to We are interested in the error probability associated to , i.e., we are interested in . To this end note that if for all in some neighborhood of zero then (13) is well-defined, see =-=[21]-=-. Therefore, if we assume that then there exists an integer such that It then follows that for this where if is a positive constant depending only on and . Now assume that for some iteration . We clai... |

125 | Linear-time encodable and decodable error-correcting codes
- Spielman
- 1996
(Show Context)
Citation Context ...s has been that their encoding complexity is high. One way to get around this problem is to slightly modify the construction of codes from bipartite graphs to a cascade of such graphs, see [2], [24], =-=[3]-=-. An alternative solution for practical purposes, which does not require cascades, is presented in [4]. Let us recall some basic notation. As originally suggested by Tanner [5], LDPC codes are well re... |

79 | Analysis of low density codes and improved designs using irregular graphs
- Luby, Mitzenmacher, et al.
- 1998
(Show Context)
Citation Context ...iable nodes have degree and all check nodes have degree . The bipartite graph determining such a code is shown in Fig. 1. Irregular LDPC codes were introduced in [2], [24] and were further studied in =-=[6]-=-–[8]. For such an irregular LDPC code, the degrees of each set of nodes are chosen according to some distribution. Thus, an irregular LDPC code might have as620 IEEE TRANSACTIONS ON INFORMATION THEORY... |

77 | Analysis of random processes via And-Or tree evaluation
- Luby, Mitzenmacher, et al.
- 1998
(Show Context)
Citation Context ...has an associated threshold, call it , it is natural to search for those pairs that maximize this threshold. 5 This was accomplished, with great success, in the case of the erasure channel [2], [24], =-=[10]-=-, [11]. For most other memoryless channels of interest the situation is much more complicated and new methods must be brought to bear on the optimization problem. Fig. 2 compares the performance of an... |

64 | New sequences of linear time erasure codes approaching the channel capacity
- Shokrollahi
(Show Context)
Citation Context ...DPC codes under belief-propagation decoding can achieve capacity over a given binary-input memoryless output-symmetric channel. 6 The only definitive results in this direction are those of [2], [24], =-=[15]-=-, which give explicit sequences of degree distribution pairs whose thresholds over the binary erasure channel (BEC) converge to the Shannon capacity limit. The following theorem, due to Gallager, impo... |

44 | Comparison of constructions of irregular Gallager codes
- MacKay, Wilson, et al.
- 1999
(Show Context)
Citation Context ...e nodes have degree and all check nodes have degree . The bipartite graph determining such a code is shown in Fig. 1. Irregular LDPC codes were introduced in [2], [24] and were further studied in [6]–=-=[8]-=-. For such an irregular LDPC code, the degrees of each set of nodes are chosen according to some distribution. Thus, an irregular LDPC code might have as620 IEEE TRANSACTIONS ON INFORMATION THEORY, VO... |

18 |
Design of efficient erasure codes with differential evolution,” Information Theory
- Shokrollahi, Storn
- 2000
(Show Context)
Citation Context ... associated threshold, call it , it is natural to search for those pairs that maximize this threshold. 5 This was accomplished, with great success, in the case of the erasure channel [2], [24], [10], =-=[11]-=-. For most other memoryless channels of interest the situation is much more complicated and new methods must be brought to bear on the optimization problem. Fig. 2 compares the performance of an insta... |

18 | Capacity-achieving sequences - Shokrollahi |

15 |
Iterative decoding and pseudocodewords
- Horn
- 1999
(Show Context)
Citation Context ...density of the received values corresponding to the channel with parameter . Then . We note that for some codes, e.g., cycle codes, the stability condition determines the threshold exactly, see [18], =-=[19]-=- for some specific examples. In this paper, we will only prove the necessity of the stated stability condition. Demonstrating sufficiency is quite involved and the proof can be found in [20]. Before v... |

8 | On the error-correcting capabilities of cycle codes of graphs
- Decreusefond, Zémor
- 1997
(Show Context)
Citation Context ...ssage density of the received values corresponding to the channel with parameter . Then . We note that for some codes, e.g., cycle codes, the stability condition determines the threshold exactly, see =-=[18]-=-, [19] for some specific examples. In this paper, we will only prove the necessity of the stated stability condition. Demonstrating sufficiency is quite involved and the proof can be found in [20]. Be... |

6 | low-density parity-check codes using irregular graphs - “Improved - 2001 |

1 |
A proof of the stability condition for LDPC codes,” paper, in preparation
- Richardson, Urbanke
(Show Context)
Citation Context ...see [18], [19] for some specific examples. In this paper, we will only prove the necessity of the stated stability condition. Demonstrating sufficiency is quite involved and the proof can be found in =-=[20]-=-. Before venturing into the proof of the necessity of the stability condition let us calculate the stability condition explicitly for various channels. Example 10 [BEC]: For the BEC (see Example 6) we... |