## Bayes Risk Weighted Vector Quantization With Posterior Estimation for Image Compression and Classification (1996)

Venue: | IEEE TRANSACTIONS ON IMAGE PROCESSING |

Citations: | 28 - 8 self |

### BibTeX

@ARTICLE{Perlmutter96bayesrisk,

author = {K. O. Perlmutter and S. M. Perlmutter and R. M. Gray and R. A. Olshen and K. L. Oehler},

title = {Bayes Risk Weighted Vector Quantization With Posterior Estimation for Image Compression and Classification},

journal = {IEEE TRANSACTIONS ON IMAGE PROCESSING},

year = {1996},

volume = {5},

pages = {347--360}

}

### Years of Citing Articles

### OpenURL

### Abstract

Classification and compression play important roles in communicating digital information. Their combination is useful in many applications, including the detection of abnormalities in compressed medical images. In view of the similarities of compression and low-level classification, it is not surprising that there are many similar methods for their design. Because some of these methods are useful for designing vector quantizers, it seems natural that vector quantization (VQ) is explored for the combined goal. We investigate several VQ-based algorithms that seek to minimize both the distortion of compressed images and errors in classifying their pixel blocks. These algorithms are investigated with both full search and tree-structured codes. We emphasize a nonparametric technique that minimizes both error measures simultaneously by incorporating a Bayes risk component into the distortion measure used for design and encoding. We introduce a tree-structured posterior estimator to produce t...

### Citations

3926 |
Classification and Regression Trees
- Breiman, Friedman, et al.
- 1984
(Show Context)
Citation Context ...e in average distortion for the given increase in average rate. The basic design algorithm is a combination of the Lloyd clustering algorithm with ideas from classification and regression tree design =-=[13]-=-, [14], [15], [2], [16], [17], [18], [19]. Just as compression can be viewed as a vectorial process, so too can be low-level classification. In particular, we consider a joint random process fX(n); Y ... |

1657 |
Vector Quantization and Signal Compression
- Gersho, Gray
- 1992
(Show Context)
Citation Context ...r of nodes in a decision tree. Because these design techniques are often design components of vector quantizers, and because vector quantization (VQ) has been applied successfully to both compression =-=[2]-=-, [3] and low-level classification [4], [5], [6], [7], [8], [9], [10], VQ is a natural technique to explore for the combined compression/classification goal. VQ is a method of lossy compression in whi... |

1309 |
Self-Organization and Associative Memory
- Kohonen
- 1989
(Show Context)
Citation Context ...e design techniques are often design components of vector quantizers, and because vector quantization (VQ) has been applied successfully to both compression [2], [3] and low-level classification [4], =-=[5]-=-, [6], [7], [8], [9], [10], VQ is a natural technique to explore for the combined compression/classification goal. VQ is a method of lossy compression in which statistical techniques are applied to op... |

1218 |
An Algorithm for Vector Quantizer Design
- Linde, Gray
- 1980
(Show Context)
Citation Context ...o constraints on bit rate and code structure. When the squared error distortion is used, the average distortion is called the mean squared error (MSE). The Lloyd or k-means clustering algorithm [11], =-=[12]-=-, [2] for full search, unconstrained VQ involves an iterative application that consists of two steps: Step 1: The optimal encoder for a given decoder is designed using a minimum distortion or nearest ... |

190 |
The divergence and Bhattacharyya distance measures in signal selection
- Kailath
- 1967
(Show Context)
Citation Context ...jx) = EL [log PL (jjX)sP (jjX) ]: If distributions are close with respect to relative entropy, then the performance of Bayes estimators and classifiers based on these distributions will also be close =-=[42]-=-. In addition, relative entropy has been applied successfully to clustering and pattern classification and it has a variety of useful geometric properties, including a simply evaluated Lloyd centroid ... |

140 |
Entropy-constrained vector quantization
- Chou, Lookabaugh, et al.
- 1989
(Show Context)
Citation Context ... distortion for the given increase in average rate. The basic design algorithm is a combination of the Lloyd clustering algorithm with ideas from classification and regression tree design [13], [14], =-=[15]-=-, [2], [16], [17], [18], [19]. Just as compression can be viewed as a vectorial process, so too can be low-level classification. In particular, we consider a joint random process fX(n); Y (n); n = 0; ... |

137 |
Neural networks and related methods for classification (with discussion
- Ripley
- 1994
(Show Context)
Citation Context ...rmits small regions of an image to be classified automatically as part of compression. It thus uses small pixel blocks to classify, similar to techniques such as neural nets and discriminant analysis =-=[33]-=- operating directly on the original image. Consequently, the classification does not require additional bits beyond those required for compression alone. With a recent technique introduced by Oehler e... |

115 | Optimal pruning with applications to tree-structured source coding and modeling - Chou, Lookabaugh, et al. - 1989 |

100 |
Statistical pattern recognition with neural network: Benchmarking studies
- Kohonen, Barna, et al.
- 1988
(Show Context)
Citation Context ...ign techniques are often design components of vector quantizers, and because vector quantization (VQ) has been applied successfully to both compression [2], [3] and low-level classification [4], [5], =-=[6]-=-, [7], [8], [9], [10], VQ is a natural technique to explore for the combined compression/classification goal. VQ is a method of lossy compression in which statistical techniques are applied to optimiz... |

86 | LVQ_PAK: the learning vector quantization program package
- Kohonen
- 1993
(Show Context)
Citation Context ...en so, minimum mean squared error encoding is used to select a representative from the codebook. The LVQ variation we consider in this study is based on Kohonen's optimized learning rate LVQ1 (olvq1) =-=[46]-=-. The algorithm encodes each training vector in succession by mapping it to the index of the nearest codeword in the codebook. The codeword is then updated in a manner that is dependent upon whether t... |

67 | Novel cluster-based probability model for texture synthesis, classification, and compression
- Popat, Picard
- 1993
(Show Context)
Citation Context ...] and Wesel et al. [28] have examined such a method for the joint goal. Another is to use a classification scheme such as Stone's generalized nearest neighbors [29], clustered nearest neighbors [30], =-=[31]-=-, [32], or Kohonen's learning VQ (LVQ) [5], [6] to classify, and then report the selected nearest neighbor (which implies the class) as the compressed reproduction. A slight modification of this appro... |

58 |
Classified Vector Quantization of Image
- Ramanurthi, Gersho
- 1986
(Show Context)
Citation Context ...in which classification is performed first, and then quantization is performed on the results of the classifier. One approach is to design separate quantizers for each class, yielding a classified VQ =-=[27]-=-. This technique has been investigated primarily for the sole objective of improved compression, although Gorman [1] and Wesel et al. [28] have examined such a method for the joint goal. Another is to... |

50 |
An introduction to neural Computing
- Kohonen
- 1988
(Show Context)
Citation Context ... these design techniques are often design components of vector quantizers, and because vector quantization (VQ) has been applied successfully to both compression [2], [3] and low-level classification =-=[4]-=-, [5], [6], [7], [8], [9], [10], VQ is a natural technique to explore for the combined compression/classification goal. VQ is a method of lossy compression in which statistical techniques are applied ... |

46 |
Consistent nonparametric regression (with discussion
- Stone
- 1977
(Show Context)
Citation Context ...improved compression, although Gorman [1] and Wesel et al. [28] have examined such a method for the joint goal. Another is to use a classification scheme such as Stone's generalized nearest neighbors =-=[29]-=-, clustered nearest neighbors [30], [31], [32], or Kohonen's learning VQ (LVQ) [5], [6] to classify, and then report the selected nearest neighbor (which implies the class) as the compressed reproduct... |

42 |
Combining image compression and classification using vector quantization
- OEHLER, GRAY
- 1995
(Show Context)
Citation Context ...we are interested in producing estimates of both the observed value X and its class Y , i.e., we wish to obtain ( X; Y ). We thus seek to minimize both MSE and Bayes risk within the VQ framework [8], =-=[20]-=-, [21]. We refer to a VQ system that addresses this goal of minimizing both MSE and Bayes risk as a Bayes risk weighted VQ or Bayes VQ in the general sense. The term will also be used more narrowly to... |

41 | Consistency of data-driven histogram methods for density estimation and classification
- Lugosi, Nobel
- 1996
(Show Context)
Citation Context ...rst the quantizer to minimize MSE and then the classifier to minimize Bayes risk. This will provide performance near that of the ideal Bayes classifier provided the training set is sufficiently large =-=[23]-=-, [24], [25], [13], [26], [17], [18], [19]. Both compression and classification can also be obtained using a sequential classifier/quantizer design, in which classification is performed first, and the... |

35 |
A greedy tree growing algorithm for the design of variable rate vector quantizers
- Riskin, Gray
- 1991
(Show Context)
Citation Context ... for the given increase in average rate. The basic design algorithm is a combination of the Lloyd clustering algorithm with ideas from classification and regression tree design [13], [14], [15], [2], =-=[16]-=-, [17], [18], [19]. Just as compression can be viewed as a vectorial process, so too can be low-level classification. In particular, we consider a joint random process fX(n); Y (n); n = 0; 1; : : :g, ... |

35 |
Applications of Ali-Silvey distance measures in the design of generalized quantizers for binary decision systems
- Poor, Thomas
- 1977
(Show Context)
Citation Context ...and classifier are designed and used separately and independently. An alternative approach is a sequential quantizer/classifier design. Such a system was investigated primarily for detection purposes =-=[22]-=-, although McLean [9] has examined the combined goal of compression and texture classification. The joint goal can be achieved by designing first the quantizer to minimize MSE and then the classifier ... |

30 |
Histogram regression estimation using data-dependent partitions
- Nobel
- 1996
(Show Context)
Citation Context ...e quantizer to minimize MSE and then the classifier to minimize Bayes risk. This will provide performance near that of the ideal Bayes classifier provided the training set is sufficiently large [23], =-=[24]-=-, [25], [13], [26], [17], [18], [19]. Both compression and classification can also be obtained using a sequential classifier/quantizer design, in which classification is performed first, and then quan... |

30 |
Global convergence and empirical consistency of the generalized Lloyd algorithm
- Sabine, Gray
- 1986
(Show Context)
Citation Context ...ng in t and bounded below by 0, J (t) must converge as t ! 1. Thus, this algorithm converges, as would any VQ algorithm using the generalized Lloyd algorithm, but not necessarily to a unique codebook =-=[38]-=-, [25]. For the design of the tree-structured Bayes VQ, the tree is grown using the usual methods applied to the modified distortion measure. For example, one begins with the optimum zero rate VQ, spl... |

23 |
Vector quantization technique for nonparametric classifier design
- Xie, Laszlo, et al.
- 1993
(Show Context)
Citation Context ...man [1] and Wesel et al. [28] have examined such a method for the joint goal. Another is to use a classification scheme such as Stone's generalized nearest neighbors [29], clustered nearest neighbors =-=[30]-=-, [31], [32], or Kohonen's learning VQ (LVQ) [5], [6] to classify, and then report the selected nearest neighbor (which implies the class) as the compressed reproduction. A slight modification of this... |

17 |
Applications of information theory to pattern recognition and the desing of decision trees and trellises
- Chou
- 1988
(Show Context)
Citation Context .... In addition, relative entropy has been applied successfully to clustering and pattern classification and it has a variety of useful geometric properties, including a simply evaluated Lloyd centroid =-=[43, 44]-=-. It can be shown that D(PL jjsP ) = H(Y jff(X)) \Gamma H(Y jX) = I(Y ; X jff(X)); the conditional mutual information between the class label and the input vector given the quantized value of the inpu... |

15 |
Minimum cross-entropy pattern classification
- Shore, Gray
- 1982
(Show Context)
Citation Context .... In addition, relative entropy has been applied successfully to clustering and pattern classification and it has a variety of useful geometric properties, including a simply evaluated Lloyd centroid =-=[43, 44]-=-. It can be shown that D(PL jjsP ) = H(Y jff(X)) \Gamma H(Y jX) = I(Y ; X jff(X)); the conditional mutual information between the class label and the input vector given the quantized value of the inpu... |

11 |
Variable Rate Vector Quantization of Images
- Riskin
- 1990
(Show Context)
Citation Context ...er to the output of the tree-structured posterior estimator. Separate full search VQs are designed for each class in a jointly greedy manner. Bits are then deallocated optimally between the codebooks =-=[45]-=- in order to attain the desired rate. Sequential classifier/TSVQ: The classifier is designed in the same manner as in the previous design. Separate TSVQs are designed for each class. The codebooks are... |

11 |
Tree-structured vector quantization of CT chest scans: image quality and diagnostic accuracy
- Cosman, Tseng, et al.
- 1993
(Show Context)
Citation Context ...mages; the additional tumor training vectors were added because of the low average percentage of tumor vectors in the data. The tumors in the training sequence were identified by trained radiologists =-=[47]-=-; 99.85% of the training vectors were not tumor vectors. We consider a two class problem in which nontumors are assigned class 0 and tumors are assigned class 1. We assign different costs (as will be ... |

10 |
Vector quantization for texture classification
- McLean
- 1993
(Show Context)
Citation Context ...are often design components of vector quantizers, and because vector quantization (VQ) has been applied successfully to both compression [2], [3] and low-level classification [4], [5], [6], [7], [8], =-=[9]-=-, [10], VQ is a natural technique to explore for the combined compression/classification goal. VQ is a method of lossy compression in which statistical techniques are applied to optimize distortion /b... |

10 | Cluster-based probability model applied to image restoration and compression
- Popat, Picard
- 1994
(Show Context)
Citation Context ...Wesel et al. [28] have examined such a method for the joint goal. Another is to use a classification scheme such as Stone's generalized nearest neighbors [29], clustered nearest neighbors [30], [31], =-=[32]-=-, or Kohonen's learning VQ (LVQ) [5], [6] to classify, and then report the selected nearest neighbor (which implies the class) as the compressed reproduction. A slight modification of this approach (i... |

9 |
Least squared quantization in PCM." unpublished memorandum
- Lloyd
- 1957
(Show Context)
Citation Context ...ject to constraints on bit rate and code structure. When the squared error distortion is used, the average distortion is called the mean squared error (MSE). The Lloyd or k-means clustering algorithm =-=[11]-=-, [12], [2] for full search, unconstrained VQ involves an iterative application that consists of two steps: Step 1: The optimal encoder for a given decoder is designed using a minimum distortion or ne... |

8 | Combining image classification and image compression using vector quantization - Oehler, Gray - 1993 |

4 |
Vector Quantization. IEEE Reprint Collection
- Abut, editor
- 1990
(Show Context)
Citation Context ...nodes in a decision tree. Because these design techniques are often design components of vector quantizers, and because vector quantization (VQ) has been applied successfully to both compression [2], =-=[3]-=- and low-level classification [4], [5], [6], [7], [8], [9], [10], VQ is a natural technique to explore for the combined compression/classification goal. VQ is a method of lossy compression in which st... |

4 |
Classification using vector quantization
- Oehler, Cosman, et al.
- 1991
(Show Context)
Citation Context ...ques are often design components of vector quantizers, and because vector quantization (VQ) has been applied successfully to both compression [2], [3] and low-level classification [4], [5], [6], [7], =-=[8]-=-, [9], [10], VQ is a natural technique to explore for the combined compression/classification goal. VQ is a method of lossy compression in which statistical techniques are applied to optimize distorti... |

4 |
Compression and Classification of Images using Vector Quantization and Decision Trees
- Perlmutter
- 1995
(Show Context)
Citation Context ...earch VQ with posterior estimation identified over 20% more of the tumor vectors correctly and provided 2.1 dB improvement in compression to that obtained with the centroid-based LVQ. Our simulations =-=[48]-=- indicated that the centroid-based LVQ provides up to 1 dB improvement in compression over the LVQ design without the centroid modification. Bayes full search VQ with posterior estimation provided bet... |

3 | Cluster compression algorithm: a joint clustering/data compression concept - Hilbert - 1977 |

2 | Bayes risk weighted VQ and learning VQ
- Wesel, Gray
- 1994
(Show Context)
Citation Context ... separate quantizers for each class, yielding a classified VQ [27]. This technique has been investigated primarily for the sole objective of improved compression, although Gorman [1] and Wesel et al. =-=[28]-=- have examined such a method for the joint goal. Another is to use a classification scheme such as Stone's generalized nearest neighbors [29], clustered nearest neighbors [30], [31], [32], or Kohonen'... |

2 |
Probability Estimation for Classification Trees and Sequence Analysis
- Walker
- 1992
(Show Context)
Citation Context ...suboptimal MSE encoder was used. We would expect the performance in the nonparametric case to be improved by incorporating good estimates of the posterior class probabilities outside the training set =-=[39]-=-, [40]. We seek to use a posterior estimator that is both effective in producing estimates that yield good overall performance and simple in terms of computational complexity. The goal of low computat... |

1 |
Vector quantizer designs for joint compression and terrain categorization of multispectral imagery
- Gorman
- 1994
(Show Context)
Citation Context ...o useful for multispectral or aerial imagery. Such imagery often entails large quantities of data that must be compressed for archival or transmission purposes and categorized into different terrains =-=[1]-=-. The goals of compression and classification are quite similar in that each can be described in the framework of the other. Compression, for example, can be viewed as a form of classification since i... |

1 |
Tree structured neural net classifier
- Kramer
- 1990
(Show Context)
Citation Context ...echniques are often design components of vector quantizers, and because vector quantization (VQ) has been applied successfully to both compression [2], [3] and low-level classification [4], [5], [6], =-=[7]-=-, [8], [9], [10], VQ is a natural technique to explore for the combined compression/classification goal. VQ is a method of lossy compression in which statistical techniques are applied to optimize dis... |

1 |
Learning-based pattern classification using vector quantization
- Skouson
- 1994
(Show Context)
Citation Context ...ften design components of vector quantizers, and because vector quantization (VQ) has been applied successfully to both compression [2], [3] and low-level classification [4], [5], [6], [7], [8], [9], =-=[10]-=-, VQ is a natural technique to explore for the combined compression/classification goal. VQ is a method of lossy compression in which statistical techniques are applied to optimize distortion /bit rat... |

1 |
Olshen, "Almost surely consistent nonparametric regression from recursive partitioning schemes
- Gordon, A
- 1984
(Show Context)
Citation Context ...he given increase in average rate. The basic design algorithm is a combination of the Lloyd clustering algorithm with ideas from classification and regression tree design [13], [14], [15], [2], [16], =-=[17]-=-, [18], [19]. Just as compression can be viewed as a vectorial process, so too can be low-level classification. In particular, we consider a joint random process fX(n); Y (n); n = 0; 1; : : :g, where ... |

1 |
Olshen, "Consistent nonparametric regression from recursive partitioning schemes
- Gordon, A
- 1980
(Show Context)
Citation Context ...en increase in average rate. The basic design algorithm is a combination of the Lloyd clustering algorithm with ideas from classification and regression tree design [13], [14], [15], [2], [16], [17], =-=[18]-=-, [19]. Just as compression can be viewed as a vectorial process, so too can be low-level classification. In particular, we consider a joint random process fX(n); Y (n); n = 0; 1; : : :g, where the X(... |

1 |
Olshen, "Asymptotically efficient solutions to the classification problem
- Gordon, A
- 1978
(Show Context)
Citation Context ...rease in average rate. The basic design algorithm is a combination of the Lloyd clustering algorithm with ideas from classification and regression tree design [13], [14], [15], [2], [16], [17], [18], =-=[19]-=-. Just as compression can be viewed as a vectorial process, so too can be low-level classification. In particular, we consider a joint random process fX(n); Y (n); n = 0; 1; : : :g, where the X(n) are... |

1 |
Olshen, "Termination and continuity of greedy growing for tree structured vector quantizers
- Nobel, A
(Show Context)
Citation Context ...tizer to minimize MSE and then the classifier to minimize Bayes risk. This will provide performance near that of the ideal Bayes classifier provided the training set is sufficiently large [23], [24], =-=[25]-=-, [13], [26], [17], [18], [19]. Both compression and classification can also be obtained using a sequential classifier/quantizer design, in which classification is performed first, and then quantizati... |

1 |
Recursive partitioning to reduce distortion", Beckman institute technical report
- Nobel
- 1995
(Show Context)
Citation Context ...imize MSE and then the classifier to minimize Bayes risk. This will provide performance near that of the ideal Bayes classifier provided the training set is sufficiently large [23], [24], [25], [13], =-=[26]-=-, [17], [18], [19]. Both compression and classification can also be obtained using a sequential classifier/quantizer design, in which classification is performed first, and then quantization is perfor... |

1 |
Olshen, "Combining tree-structured vector quantization with classification and regression trees
- Gray, Oehler, et al.
- 1993
(Show Context)
Citation Context ...er. If these probabilities are known and can be used in both the design and encoding, then the performance of the algorithm can be improved. In the parametric case, for example, it was shown in [20], =-=[36]-=- that the incorporation of these probabilities into the encoding process can improve performance significantly. It is thus desirable to estimate the posterior probabilities so that they can be used in... |

1 |
Probability estimation for classification trees", Submitted for publication
- Walker, Olshen, et al.
- 1994
(Show Context)
Citation Context ...imal MSE encoder was used. We would expect the performance in the nonparametric case to be improved by incorporating good estimates of the posterior class probabilities outside the training set [39], =-=[40]-=-. We seek to use a posterior estimator that is both effective in producing estimates that yield good overall performance and simple in terms of computational complexity. The goal of low computational ... |

1 |
Predicting induced duodenal ulcer and adrenal necrosis with classification trees
- Giampaolo, Gray, et al.
- 1991
(Show Context)
Citation Context ...t new; it has been explored by Xie et al. [30], Popat and Picard [32], and Nobel [24], and the use of tree-structured methods for classification has been investigated in [13], [17], [19], [39], [40], =-=[41]-=-. We construct the TSVQ from empirical distributions of the training data used to design the Bayes VQ. The estimate of the posterior probability is subsequently determined by the relative frequencies ... |