The history of the theory and practice of quantization dates to 1948, although similar ideas had appeared in the literature as long ago as 1898. The fundamental role of quantization in modulation and analog-to-digital conversion was first recognized during the early development of pulsecode modulation systems, especially in the 1948 paper of Oliver, Pierce, and Shannon. Also in 1948, Bennett published the first high-resolution analysis of quantization and an exact analysis of quantization noise for Gaussian processes, and Shannon published the beginnings of rate distortion theory, which would provide a theory for quantization as analog-to-digital conversion and as data compression. Beginning with these three papers of fifty years ago, we trace the history of quantization from its origins through this decade, and we survey the fundamentals of the theory and many of the popular and promising techniques for quantization.

Entropy-coded vector quantization is studied using high-resolution multidimensional companding over a class of nondifference distortion measures. For distortion measures which are "locally quadratic" a rigorous derivation of the asymptotic distortion and entropy-coded rate of multidimensional companders is given along with conditions for the optimal choice of the compressor function. This optimum compressor, when it exists, depends on the distortion measure but not on the source distribution. The rate-distortion performance of the companding scheme is studied using a recently obtained asymptotic expression for the rate-distortion function which parallels the Shannon lower bound for difference distortion measures. It is proved that the high-resolution performance of the scheme is arbitrarily close to the rate-distortion limit for large quantizer dimensions if the compressor function and the lattice quantizer used in the companding scheme are optimal, extending an analogous statement for...

High-resolution bounds in lossy coding of a real memoryless source are considered when side information is present. Let be a "smooth" source and let be the side information. First we treat the case when both the encoder and the decoder have access to and we establish an asymptotically tight (high-resolution) formula for the conditional rate-distortion function ( ) for a class of locally quadratic distortion measures which may be functions of the side information. We then consider the case when only the decoder has access to the side information (i.e., the "Wyner--Ziv problem"). For side-information-dependent distortion measures, we give an explicit formula which tightly approximates the Wyner--Ziv rate-distortion function ( ) for small under some assumptions on the joint distribution of and . These results demonstrate that for side-information-dependent distortion measures the rate loss ( ) ( ) can be bounded away from zero in the limit of small . This contrasts the case of distortion measures which do not depend on the side information where the rate loss vanishes as 0.

Consider encoding a source X into two descriptions, such that the first, the second and both descriptions allow decoding of X with distortion levels d 1 , d 2 and d 0 , respectively, relative to a distortion measure ae(x; x). Ozarow have found an explicit characterization for the region R (oe 2 ; d 1 ; d 2 ; d 0 ) of admissible rate pairs of the two descriptions, for a Gaussian source X ¸ N (0; oe 2 ), relative to the squared-error distortion measure ae(x; x) = (x \Gamma x) 2 . In fact, this is the only case for which the multiple description rate-distortion region is completely known. We show that for a general real valued source, a locally quadratic distortion measure of the form ae(x; x) = w(x) 2 (x \Gamma x) 2 + o((x \Gamma x) 2 ), and small distortion levels, the region of admissible rate pairs equals approximately R i P x 2 2Eflog w(X)g ; d 1 ; d 2 ; d 0 j where P x is the entropy-power of the source. Applications to companding quantization are a...

High resolution bounds in lossy coding of a real memoryless source are considered when side information is present. Let X be a "smooth" source and let Y be the side information. First we treat the case when both the encoder and the decoder have access to Y and we establish an asymptotically tight (high-resolution) formula for the conditional rate-distortion function R XjY (D) for a class of locally quadratic distortion measures which may be functions of the side information. We then consider the case when only the decoder has access to the side information (i.e., the "Wyner-Ziv problem"). For side information dependent distortion measures, we give an explicit formula which tightly approximates the Wyner-Ziv rate-distortion function R WZ (D) for small D under rather general assumptions on the joint distribution of X and Y . These results demonstrate that for side information dependent distortion measures the rate loss R WZ (D) \Gamma R XjY (D) can be bounded away from zero in th...

The connection between compression and the estimation of probability distributions has long been known for the case of discrete alphabet sources and lossless coding. A universal lossless code which does a good job of compressing must implicitly also do a good job of modeling. In particular, with a collection of codebooks, one for each possible class or model, if codewords are chosen from among the ensemble of codebooks so as to minimize bit rate, then the codebook selected provides an implicit estimate of the underlying class. Less is known about the corresponding connections between lossy compression and continuous sources. Here we consider aspects of estimating conditional and unconditional densities in conjunction with Bayes-risk weighted vector quantization for joint compression and classification.

We investigate high rate quantization for various detection and reconstruction loss critera. A new distortion measure...

In this paper, we derive an asymptotically optimal multi-layer coding scheme for entropy-coded scalar quantizers (S0) that minimizes the weighted mean-squared error (WMSE). The optimal entropy-coded S0 is non-uniform in the case of WMSE. The conventional multi-layer coder quantizes the base-layer reconstruction error at the enhancement-layer, and is sub-optimal for the WMSE criterion. We consider the cornpander representation of the quantizer, and propose to implement scalability in the compressed domain. We show that such a multi-layer coding system achieves the operational rate-distortion bound given by the non-scalable entropy-coded S0, at the limit of high resolution. Simulation results for a synthetic memoryless Laplace source with/-law companding are presented for various values of layer rates. Substantial gains are also achieved on the "real-world" sources of audio signals, when the optimal multi-layer approach is applied to a two-layer scalable MPEG-4 Advanced Audio Coder.

In emission tomographic modalities such as SPECT or PET various regions of the detector space yield different amounts of information about the emission source. This paper develops a framework for exploring the tradeoffs between binning the detector space to reduce memory storage (complexity) and the associated loss in performance for image reconstruction tasks and detection tasks. We use high rate vector quantization theory to establish just how much relevant information can be preserved after compression of the emission measurement data. We illustrate our results for one dimensional deconvolution and two dimensional lesion detection in PET.

We propose a new approach to achieve efficient scalability in audio coders, and demonstrate its performance using the MPEG-4 Advanced Audio Coder (AAC). In conventional scalable coding, the enhancement-layer performs straightforward re-quantization of the base-layer reconstruction error. This coding scheme implicitly discards useful information from the base-layer, and does not truly minimize a perceptually meaningful distortion criterion such as the noise-mask ratio. We reformulate the problem of scalable coding within a companding framework, and show that re-quantization in the compander's compressed domain achieves, in the asymptotic sense, optimal scalability. Based on this observation, we develop a scalable AAC coder which performs enhancement-layer quantization while exploiting all the information available at that layer. Simulation results of a two-layer scalable coder on the standard test database of 44.1kHz sampled audio show that the proposed approach yields substantial savings in bit rate for a given reproduction quality.