Today's digital libraries increasingly include not only printed text but also scanned handwritten pages and other multimedia material. There are, however, few tools available for manipulating handwritten pages. Here, we propose an algorithm based on dynamic time warping (DTW) for a word by word alignment of handwritten documents with their (ASCII) transcripts. We see at least three uses for such alignment algorithms. First, alignment algorithms allow us to produce displays (for example on the web) that allow a person to easily find their place in the manuscript when reading a transcript. Second, such alignment algorithms will allow us to produce large quantities of ground truth data for evaluating handwriting recognition algorithms. Third, such algorithms allow us to produce indices in a straightforward manner for handwriting material. We provide experimental results of our algorithm on a set of 70 pages of historical handwritten material - specifically the writings of George Washington. Our method achieves 74.5% accuracy on line by line alignment and 60.5% accuracy when aligning whole pages at time.

A key problem faced by classifiers is coping with styles not represented in the training set. We present an application of hierarchical Bayesian methods to the problem of recognizing degraded printed characters in a variety of fonts. The proposed method works by using training data of various styles and classes to compute prior distributions on the parameters for the class conditional distributions. For classification, the parameters for the actual class conditional distributions are fitted using an EM algorithm. The advantage of hierarchical Bayesian methods is motivated with a theoretical example. Severalfold increases in classification performance relative to style-oblivious and style-conscious are demonstrated on a multifont OCR task.

When patterns occur in large groups generated by a single source (style consistent test data), the statistics of the test data differ from those of the training data which consists of patterns from all sources. We present a Gaussian model for continuously distributed sources under which we develop adaptive classifiers that specialize to the statistics of styleconsistent test data. On NIST handwritten digit data, the adaptive classifiers reduce the error rate by more than 50% operating on one writer (#10 samples/class) at a time.

OCR systems for printed documents typically require large numbers of font styles and character models to work well. When given an unseen font, performance degrades even in the absence of noise. In this paper, we perform OCR in an unsupervised fashion without using any character models by using a cryptogram decoding algorithm. We present results on real and artificial OCR data.

We present strategies and results for identifying the symbol type (lower-case, upper-case, digit, and punctuation or special symbols) of every character in a text document by using various kinds of information from neighboring characters. In the expectation of reasonable word and character segmentation for shape clustering, we designed several type recognition methods that depend on cluster n-grams, shape codes, and withinword context. On an ASCII test corpus of 925 articles that simulates perfect image-level processing, these methods achieve a substantial improvement over default assignment of all characters to lower case.

The fundamental task facing Optical Character Recognition (OCR) systems involves the conversion of input document images into corresponding sequences of symbolic character codes. Traditionally, this has been accomplished in a bottom-up fashion: the image of each symbol is isolated, then classified based on its pixel intensities. While such shape-based classifiers are initially trained on a wide array of fonts, they still tend to perform poorly when faced with novel glyph shapes. In this thesis, we attempt to bypass this problem by pursuing a top-down “codebreaking ” approach. We assume no a priori knowledge of character shape, instead relying on statistical information and language constraints to determine an appropriate character mapping. We introduce and contrast three new top-down approaches, and present experimental results on several real and synthetic datasets. Given sufficient amounts of data, our font and shape independent approaches are shown to perform about as well as shape-based classifiers. ii Acknowledgements First and foremost, I would like to thank my supervisor Sam Roweis for his tireless sup-

We present strategies and results for identifying the symbol type of every character in a text document. Assuming reasonable word and character segmentation for shape clustering, we designed several type recognition methods that depend on cluster n-grams, characteristics of neighbors, and within-word context. On an ASCII test corpus of 925 articles, these methods represent a substantial improvementover default assignmentofallcharacters to lower case.

Two methods, Symbolic Indirect Correlation (SIC) and

We consider models for which it is important, early in processing, to estimate some variables with high precision, but perhaps at relatively low rates of recall. If some variables can be identified with near certainty, then they can be conditioned upon, allowing further inference to be done efficiently. Specifically, we consider optical character recognition (OCR) systems that can be bootstrapped by identifying a subset of correctly translated document words with very high precision. This “clean set ” is subsequently used as document-specific training data. While many current OCR systems produce measures of confidence for the identity of each letter or word, thresholding these confidence values, even at very high values, still produces some errors. We introduce a novel technique for identifying a set of correct words with very high precision. Rather than estimating posterior probabilities, we bound the probability that any given word is incorrect under very general assumptions, using an approximate worst case analysis. As a result, the parameters of the model are nearly irrelevant, and we are able to identify a subset of words, even in noisy documents, of which we are highly confident. On our set of 10 documents, we are able to identify about 6 % of the words on average without making a single error. This ability to produce word lists with very high precision allows us to use a family of models which depends upon such clean word lists. 1

Despite ubiquitous claims that optical character recognition (OCR) is a “solved problem, ” many categories of documents continue to break modern OCR software such as documents with moderate degradation or unusual fonts. Many approaches rely on pre-computed or stored character models, but these are vulnerable to cases when the font of a particular document was not part of the training set, or when there is so much noise in a document that the font model becomes weak. To address these difficult cases, we present a form of iterative contextual modeling that learns character models directly from the document it is trying to recognize. We use these learned models both to segment the characters and to recognize them in an incremental, iterative process. We present results comparable to those of a commercial OCR system on a subset of characters from a difficult test document. 1.