The history of the theory and practice of quantization dates to 1948, although similar ideas had appeared in the literature as long ago as 1898. The fundamental role of quantization in modulation and analog-to-digital conversion was first recognized during the early development of pulsecode modulation systems, especially in the 1948 paper of Oliver, Pierce, and Shannon. Also in 1948, Bennett published the first high-resolution analysis of quantization and an exact analysis of quantization noise for Gaussian processes, and Shannon published the beginnings of rate distortion theory, which would provide a theory for quantization as analog-to-digital conversion and as data compression. Beginning with these three papers of fifty years ago, we trace the history of quantization from its origins through this decade, and we survey the fundamentals of the theory and many of the popular and promising techniques for quantization.

This paper investigates the use of Gaussian Selection (GS) to increase the speed of a large vocabulary speech recognition system. Typically 30-70% of the computational time of a continuous density HMM-based speech recogniser is spent calculating probabilities. The aim of GS is to reduce this load by selecting the subset of Gaussian component likelihoods that should be computed given a particular input vector. This paper examines new techniques for obtaining "good" Gaussian subsets or "shortlists". All the new schemes make use of state information, specifically which state each of the Gaussian components belongs to. In this way a maximum number of Gaussian components per state may be specified, hence reducing the size of the shortlist. The first technique introduced is a simple extension of the standard GS method, which uses this state information. Then, more complex schemes based on maximising the likelihood of the training data are proposed. These new approaches are compared with the standard GS scheme on a large vocabulary speech recognition task. On this task, the use of state information reduced the percentage of Gaussians computed to 10-15%, compared with 20-30% for the standard GS scheme, with little degradation in performance. 1 M.J.F.Gales is now at the IBM T.J. Watson Research Center, Yorktown Heights, NY 10598, USA. 2 K.M. Knill is now at Nuance Communications, 1380 Willow Rd, Menlo Park, CA 94025, USA. List of Tables 1 Change in the average forced alignment likelihood of the ARPA 1994 H1 development data for SGS and SBGS systems, compared to the standard no GS system. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2 Recognition performance of the standard no GS, SGS and SBGS systems on the ARPA 1994 H...

A new fast nearest neighbor algorithm is described that uses principal component analysis to build an efficient search tree. At each node in the tree, the data set is partitioned along the direction of maximum variance. The search algorithm efficiently uses a depth-first-search and a new elimination criterion. The new algorithm was compared to sixteen other fast nearest neighbor algorithms on three types of common benchmark data sets including problems from time series prediction and image vector quantization. This comparative study illustrates the strengths and weaknesses of all of the leading algorithms. The new algorithm performed very well on all of the data sets and was consistently ranked among the top three algorithms.

In this paper, we provide an overview of fast nearest-neighbor search algorithms based on an &approxima- tion}elimination' framework under a class of elimination rules, namely, partial distance elimination, hypercube elimination and absolute-error-inequality elimination derived from approximations of Euclidean distance. Previous algorithms based on these elimination rules are reviewed in the context of approximation}elimination search. The main emphasis in this paper is a comparative study of these elimination constraints with reference to their approximation}elimination e$ciency set within di!erent approximation schemes. # 2000 Pattern Recognition Society. Published by Elsevier Science Ltd. All rights reserved.

This paper investigates the use of Gaussian Selection (GS) to reduce the state likelihood computation in HMM-based systems. These likelihood calculations contribute significantly (30 to 70%) to the computational load. Previously, it has been reported that when GS is used on large systems the recognition accuracy tends to degrade above a \Theta3 reduction in likelihood computation. To explain this degradation, this paper investigates the trade-offs necessary between achieving good state likelihoods and low computation. In addition, the problem of unseen states in a cluster is examined. It is shown that further improvements are possible. For example, using a different assignment measure, with a constraint on the number of components per state per cluster, enabled the recognition accuracy on a 5k speaker-independent task to be maintained up to a \Theta5 reduction in likelihood computation. 1. INTRODUCTION In recent years, high accuracy large vocabulary continuous speech recognition sys...

A fundamental activity common to many image processing, pattern classification, and clustering algorithms involves searching a set of n, k-dimensional data for the one which is nearest to a given target item with respect to a distance function. Our goal is to find fast search algorithms which are full-search equivalent---that is, the resulting match is as good as what we could obtain if we were to search the set exhaustively. We propose a framework made up of three components, namely (i) a technique for obtaining a good initial match, (ii) an inexpensive method for determining whether the current match is a full-search equivalent match, and (iii) an effective technique for improving the current match. Our approach is to consider good solutions for each component in order to find an algorithm which balances the overall complexity of the search. We also propose a technique for hierarchical ordering and cluster elimination using a minimal cost spanning tree. Our experiments on vector quantisation coding of images show that the framework and techniques we proposed can be used to construct suitable algorithms for most of our data sets which require full-search equivalent matches at an average arithmetic cost of less than O(k log n) while using only O(n) space.

The codeword searching sequence is sometimes very vital to the efficiency of a VQ encoding algorithm. In this paper, we evaluate some necessary criteria for the derivation of an optimal searching sequence and derive the optimal searching sequence based on such criteria. Keywords--- I. Introduction T WO common strategies have been used to reduce the complexity inherent in Vector Quantization (VQ) encoding algorithms. One resorts to simpler but suboptimal variants and sacrifices quality such as the tree searched VQ[1]. The other remains with the original VQ and devises fast algorithms such as the partial distance search (PDS). This second category of algorithms is more flexible since they are codebook-independent[2-8], but the searching sequence of the codewords is very vital to the efficiency of the algorithms. Consider the case that one has to represent a given Ddimensional input vector ~x = (x 1 ; x 2 :::x D ) with a particular codeword selected from a codebook containing N codew...

This paper investigates the use of Gaussian Selection (GS) to reduce the state likelihood computation in HMM-based systems. These likelihood calculations contribute significantly (30 to 70%) to the computational load. Previously, it has been reported that when GS is used on large systems the recognition accuracy tends to degrade above a #3 reduction in likelihood computation. To explain this degradation, this paper investigates the trade-offs necessary between achieving good state likelihoods and low computation. In addition, the problem of unseen states in a cluster is examined. It is shown that further improvements are possible. For example, using a different assignment measure, with a constraint on the number of components per state per cluster, enabled the recognition accuracy on a 5k speaker-independent task to be maintained up to a #5 reduction in likelihood computation.