## Low density codes achieve the ratedistortion bound (2006)

Venue: | In Data Compression Conference |

Citations: | 21 - 9 self |

### BibTeX

@INPROCEEDINGS{Martinian06lowdensity,

author = {Emin Martinian and Martin Wainwright},

title = {Low density codes achieve the ratedistortion bound},

booktitle = {In Data Compression Conference},

year = {2006},

pages = {153--162}

}

### Years of Citing Articles

### OpenURL

### Abstract

Abstract: We propose a new construction for low-density source codes with multiple parameters that can be tuned to optimize the performance of the code. In addition, we introduce a set of analysis techniques for deriving upper bounds for the expected distortion of our construction, as well as more general low-density constructions. We show that (with an optimal encoding algorithm) our codes achieve the rate-distortion bound for a binary symmetric source and Hamming distortion. Our methods also provide rigorous upper bounds on the minimum distortion achievable by previously proposed low-density constructions. 1

### Citations

8603 |
Elements of information theory
- Cover, Thomas
- 1991
(Show Context)
Citation Context ... a randomly generated code C, then the probability that the code is successful is simply 2 Pr[z(D) > 0]. Since analyzing this probability directly is generally difficult, most random coding arguments =-=[4]-=- consider the expectation E[z(D)]. For essentially any code (and in particular the compound construction), it is possible to show that the expected number of good codewords is large: Lemma 2. 3.2 Typi... |

1693 |
The Probabilistic Method
- Alon, Spencer
- 1992
(Show Context)
Citation Context ... correspond to the typical distortion. To prove exponential concentration, we show that the actual error probability, Pr[z(D) = 0] is smaller than e −cn for some constant c using martingale arguments =-=[1, 10]-=-. Specifically, we define a Doob martingale mi(Cb) that is the expected value of the distortion between the best codeword and the source (conditioned on the bottom code Cb) when the first i columns of... |

894 | Low-Density Parity-Check Codes
- Gallager
- 1963
(Show Context)
Citation Context ...M (top section) with γt = 4 and an LDPC (bottom section) with (γv, γc) = (2, 4). Random LDPC Ensemble: For the bottom LDPC code, we use the standard (γv, γc)-regular LDPC ensemble studied by Gallager =-=[5]-=-. Specifically, each of the m variable nodes in the middle layer connects to γv check nodes in the bottom layer. Similarly, each of the k check nodes in the bottom layer connects to γc variable nodes ... |

367 | Urbanke “The Capacity of Low-Density Parity-Check Codes Under Message-Passing Decoding
- Richardson, L
- 2001
(Show Context)
Citation Context ...nds on the minimum distortion achievable by previously proposed low-density constructions. 1 Introduction While low-density parity check (LDPC) codes can provably approach the channel coding capacity =-=[10]-=- for point-to-point transmission, currently there are relatively few theoretical results on low-density codes for lossy source coding, channel coding with encoder side information, and source coding w... |

101 |
Trellis coded quantization of memoryless and Gauss-Markov sources
- Marcellin, Fischer
- 1990
(Show Context)
Citation Context ...on. Even though quantization and error correction are closely related, the standard LDPC constructions used for channel coding generally fail [8]. One viable option is trellisbased quantization (TCQ) =-=[7]-=-, which has been used both for lossy source coding, as well as for distributed source coding [13, 2, 14]. However, saturating fundamental bounds with TCQ requires taking the constraint length to infin... |

47 |
On ensembles of low-density parity-check codes: asymptotic distance distributions
- Litsyn, Shevelev
- 2002
(Show Context)
Citation Context ...st have a weight enumerator that grows less quickly than the error exponent in (5). Using the exact formula for the asymptotic weight enumerator of regular LDPC codes developed by Litsyn and Shevelev =-=[6]-=-, it is possible to prove the following result: 4 Proposition 2. There exist choices for γt, γv, and γc such that the term in braces in (9) becomes negligible. 4 The proof essentially requires showing... |

31 | Iterative quantization using codes on graphs
- Martinian, Yedidia
- 2003
(Show Context)
Citation Context ...tter three scenarios all involve some aspect of quantization. Even though quantization and error correction are closely related, the standard LDPC constructions used for channel coding generally fail =-=[8]-=-. One viable option is trellisbased quantization (TCQ) [7], which has been used both for lossy source coding, as well as for distributed source coding [13, 2, 14]. However, saturating fundamental boun... |

28 |
Trellis encoding of memoryless discrete-time sources with a fidelity criterion
- Viterbi, Omura
- 1974
(Show Context)
Citation Context ...ich has been used both for lossy source coding, as well as for distributed source coding [13, 2, 14]. However, saturating fundamental bounds with TCQ requires taking the constraint length to infinity =-=[11]-=-, which incurs exponential complexity even for message-passing decoders/encoders. Consequently, it is of considerable interest to develop low-density constructions that are also capable of saturating ... |

24 |
Turbo and trellis-based constructions for source coding with side information
- Chou, Pradhan, et al.
(Show Context)
Citation Context ...ctions used for channel coding generally fail [8]. One viable option is trellisbased quantization (TCQ) [7], which has been used both for lossy source coding, as well as for distributed source coding =-=[13, 2, 14]-=-. However, saturating fundamental bounds with TCQ requires taking the constraint length to infinity [11], which incurs exponential complexity even for message-passing decoders/encoders. Consequently, ... |

22 |
Design of trellis codes for source coding with side information at the decoder
- Wang, Orchard
- 2001
(Show Context)
Citation Context ...ctions used for channel coding generally fail [8]. One viable option is trellisbased quantization (TCQ) [7], which has been used both for lossy source coding, as well as for distributed source coding =-=[13, 2, 14]-=-. However, saturating fundamental bounds with TCQ requires taking the constraint length to infinity [11], which incurs exponential complexity even for message-passing decoders/encoders. Consequently, ... |

21 |
Thouless-Anderson-Palmer approach for lossy compression
- Murayama
(Show Context)
Citation Context ...s, are provably optimal for binary erasure quantization (a special type of source coding). This motivates the use of LDGM codes and variants for more general compression problems. Indeed, recent work =-=[3, 9, 12]-=- has shown empirically that LDGM codes, in conjunction with variants of sum-product messagepassing for encoding, can approach the rate-distortion bound for a binary symmetric source (BSS). In addition... |

18 | On multiterminal source code design
- Yang, Stankovic, et al.
- 2005
(Show Context)
Citation Context ...ctions used for channel coding generally fail [8]. One viable option is trellisbased quantization (TCQ) [7], which has been used both for lossy source coding, as well as for distributed source coding =-=[13, 2, 14]-=-. However, saturating fundamental bounds with TCQ requires taking the constraint length to infinity [11], which incurs exponential complexity even for message-passing decoders/encoders. Consequently, ... |

16 |
Message-passing algorithms for non-linear nodes and data compression
- Ciliberti, Mézard, et al.
- 2005
(Show Context)
Citation Context ...s, are provably optimal for binary erasure quantization (a special type of source coding). This motivates the use of LDGM codes and variants for more general compression problems. Indeed, recent work =-=[3, 9, 12]-=- has shown empirically that LDGM codes, in conjunction with variants of sum-product messagepassing for encoding, can approach the rate-distortion bound for a binary symmetric source (BSS). In addition... |

10 |
Lossy source coding via messagepassing and decimation over generalized codewords of LDGM codes
- Wainwright, Maneva
- 2005
(Show Context)
Citation Context ...s, are provably optimal for binary erasure quantization (a special type of source coding). This motivates the use of LDGM codes and variants for more general compression problems. Indeed, recent work =-=[3, 9, 12]-=- has shown empirically that LDGM codes, in conjunction with variants of sum-product messagepassing for encoding, can approach the rate-distortion bound for a binary symmetric source (BSS). In addition... |