We study two families of error-correcting codes defined in terms of very sparse matrices. "MN" (MacKay--Neal) codes are recently invented, and "Gallager codes" were first investigated in 1962, but appear to have been largely forgotten, in spite of their excellent properties. The decoding of both codes can be tackled with a practical sum-product algorithm. We prove that these codes are "very good," in that sequences of codes exist which, when optimally decoded, achieve information rates up to the Shannon limit. This result holds not only for the binary-symmetric channel but also for any channel with symmetric stationary ergodic noise. We give experimental results for binary-symmetric channels and Gaussian channels demonstrating that practical performance substantially better than that of standard convolutional and concatenated codes can be achieved; indeed, the performance of Gallager codes is almost as close to the Shannon limit as that of turbo codes. Index Terms--- Error-correctio...

We report the empirical performance of Gallager's low density parity check codes on Gaussian channels. We show that performance substantially better than that of standard convolutional and concatenated codes can be achieved; indeed the performance is almost as close to the Shannon limit as that of Turbo codes.

Abstract—We construct new families of error-correcting codes based on Gallager’s low-density parity-check codes. We improve on Gallager’s results by introducing irregular parity-check matrices and a new rigorous analysis of hard-decision decoding of these codes. We also provide efficient methods for finding good irregular structures for such decoding algorithms. Our rigorous analysis based on martingales, our methodology for constructing good irregular codes, and the demonstration that irregular structure improves performance constitute key points of our contribution. We also consider irregular codes under belief propagation. We report the results of experiments testing the efficacy of irregular codes on both binary-symmetric and Gaussian channels. For example, using belief propagation, for rate I R codes on 16 000 bits over a binary-symmetric channel, previous low-density parity-check codes can correct up to approximately 16 % errors, while our codes correct over 17%. In some cases our results come very close to reported results for turbo codes, suggesting that variations of irregular low density parity-check codes may be able to match or beat turbo code performance. Index Terms—Belief propagation, concentration theorem, Gallager codes, irregular codes, low-density parity-check codes.

Since the invention of \turbo codes" by Berrou et al. in 1993, the \turbo principle" has been adapted to several communication problems such as \turbo equalization", \turbo trellis coded modulation", and iterative multi user detection. In this paper we study the \turbo equalization" approach, which can be applied to coded data transmission over channels with intersymbol interference (ISI). In the original system invented by Douillard et al., the data is protected by a convolutional code and a receiver consisting of two trellis-based detectors are used, one for the channel (the equalizer) and one for the code (the decoder). It has been shown that iterating equalization and decoding tasks can yield tremendous improvements in bit error rate (BER). We introduce new approaches to combining equalization based on linear ltering with the decoding. The result is a receiver that is capable of improving BER performance through iterations of equalization and decoding in a manner similar to turbo ...

Abstract-The performance of Turbo codes is addressed by examining the code’s distance spectrum. The “error floor ” that occurs at moderate signal-to-noise ratios is shown to be a conse-quence of the relatively low free distance of the code. It is also shown that the “error floor ” can be lowered by increasing the size of the interleaver without changing the free distance of the code. Alternatively, the free distance of the code may be increased by using primitive feedback polynomials. The excellent performance of lurbo codes at low signal-to-noise ratios is explained in terms of the distance spectrum. The interleaver in the Turbo encoder is shown to reduce the number of low-weight codewords through a process called “spectral thinning. ” This thinned distance spec-trum results in the free distance asymptote being the dominant performance parameter for low and moderate signal-to-noise ratios. Index Terms-Turbo codes, convolutional codes, distance spec-trum. T I.

In [6], Gallager introduces a family of codes based on sparse bipartite graphs, which he calls low-density paritycheck codes. He suggests a natural decoding algorithm for these codes, and proves a good bound on the fraction of errors that can be corrected. As the codes that Gallager builds are derived from regular graphs, we refer to them as regular codes. Following the general approach introduced in [7] for the design and analysis of erasure codes, we consider error-correcting codes based on random irregular bipartite graphs, which we call irregular codes. We introduce tools based on linear programming for designing linear time irregular codes with better error-correcting capabilities than possible with regular codes. For example, the decoding algorithm for the rate 1/2 regular codes of Gallager can provably correct up to 5.17% errors asymptotically, whereas we have found irregular codes for which our decoding algorithm can provably correct up to 6.27% errors asymptotically. We incl...

This paper addresses turbo-encoder design for coding with high spectral efficiency using parallel concatenated trellis coded modulation (PCTCM) and symbol interleaving. The turbo-encoder design involves the constituent encoder design and the interleaver design. The constituent encoders are optimized for symbol-wise effective free distance, and each has an infinite symbol-wise impulse response. We identify the canonical structures for the constituent encoder search space. In many cases of practical interest, the optimal structure for these constituent encoders connects the memory elements in a single row. This single row generally applies to turbo-code constituent encoders for parallel concatenation and is not restricted to symbol interleaving. To lower the error floor, a new semi-random interleaver design criteria and a construction method extends the spread-interleaver concept introduced by Divsalar and Pollara. Simulation results show that the proposed system employing symbol interleaving can converge at a lower SNR than previously reported systems. We report simulation results between 0.5 dB and 0.6 dB from constrained capacity for rates of 2 and 4 bits/sec/Hz.

We study the convergence behavior of turbo decoding, turbo equalization, and turbo bit-interleaved coded modulation in a unified framework, which is to regard all three principles as instances of iterative decoding of two serially concatenated codes. There is a collection of measures in the recent literature, which trace the convergence of iterative decoding algorithms based on a single parameter. This parameter is assumed to completely describe the behavior of the soft-in soft-out decoders being part of the iterative algorithm. The measures observe different parameters and were originally applied to different types of decoders. In this paper, we show how six of those measures are related to each other and we compare their convergence prediction capability for the decoding principles mentioned above. We observed that two measures predict the convergence very well for all regarded decoding principles and others suffer from systematic prediction errors independent of the decoding principle.

This paper presents a method for efficient coding at high spectral efficiency using parallel concatenated trellis coded modulation (PCTCM) with symbol interleaving. The constituent encoders are optimized for symbolwise free distance, and each has an infinite symbol-wise impulse response. In many cases of practical interest, the optimal structure for these constituent encoders connects the memory elements in a single row. Simulation results show that performance is as close as 0:5 dB to constrained capacity. I. Introduction This paper presents a method for parallel concatenated trellis coded modulation (PCTCM) with constituent encoders of rate k=n, k ? 1. The k binary inputs can be thought of as one symbol input over the extension field GF (2 k ). This approach uses one symbol interleaver between the constituent encoders instead of k bit interleavers. The use of a symbol interleaver implies that the constituent encoders should be optimized for "symbol effective free distance." This ...

We present a construction of high-rate linear error-correcting codes based on random bipartite graphs. In contrast to other researchers, our approach uses these graphs to produce generator matrices. A distributive parallel decoding algorithm based on the principle of belief propagation is presented. These new codes are then shown to be linear-time encodable and decodable. Experimental results further show the performances of these new systems to be approach the channel capacities of the Gaussian channels with binary or multilevel signal constellations. 1 Introduction The introduction of multi-stage iterative decoding with exchange of soft information [3] has recently attracted substantial research interests and reperesents a genuine breakthrough in coding theory. This methodology has been applied to parallel concatenated convolutional (turbo) codes [5, 11, 12, 17, 22], parallel concatenated block codes [17], serial concatenated convolutional codes [2], and generalized concatenated con...