We propose a new learning algorithm for regression modeling. The method is especially suitable for optimizing neural network structures that are amenable to a statistical description as mixture models. These include mixture of experts, hierarchical mixture of experts (HME), and normalized radial basis functions (NRBF). Unlike recent maximum likelihood (ML) approaches, we directly minimize the (squared) regression error. We use the probabilistic framework as means to define an optimization method that avoids many shallow local minima on the complex cost surface. Our method is based on deterministic annealing (DA), where the entropy of the system is gradually reduced, with the expected regression cost (energy) minimized at each entropy level. The corresponding Lagrangian is the system's "free-energy," and this annealing process is controlled by variation of the Lagrange multiplier, which acts as a "temperature" parameter. The new method consistently and substantially outperformed the com...

viii 1 Introduction 1 1.1 Handwritten Character Recognition : : : : : : : : : : : : : : : : : 1 1.2 Related Work : : : : : : : : : : : : : : : : : : : : : : : : : : : : : 4 1.2.1 Feature Extraction : : : : : : : : : : : : : : : : : : : : : : 4 1.2.2 Handwriting Recognition : : : : : : : : : : : : : : : : : : : 6 1.3 Proposed Approach : : : : : : : : : : : : : : : : : : : : : : : : : : 9 1.4 Thesis Organization : : : : : : : : : : : : : : : : : : : : : : : : : : 12 2 Recognition and Representation of Numeral Patterns 13 2.1 Recognition Based on Human Logical Understanding : : : : : : : 13 2.1.1 Local Geometric Shape Features : : : : : : : : : : : : : : : 14 2.1.2 Learning of Different Contributions Among Local Shape Features : : : : : : : : : : : : : : : : : : : : : : : : : : : : 17 2.1.3 Learning of New Variants by Feature Generation : : : : : 17 2.2 Invariance Based on Biological Visual System : : : : : : : : : : : 18 2.2.1 Log-Polar Transformation : : : : : : : : : : : : : : : : :...

Handwriting recognition has attracted enormous scientific interest because of its potential for improved man/machine interfaces. We have designed an on-line handwritten digit recognition system after the examination of different techniques based on statistical and neural pattern recognition approaches. We collected a digit database from 44 people. We use two representations. The dynamic representation is based on constant length feature vectors of equally distanced points on the pen trajectory. The static representation converts the dynamic information to an image similar to images used in off-line recognition tasks.Then, we tested the well known statistical classification method k-nearest neighbor (k-NN) and neural multi-layer perceptron (MLP) and recurrent networks using both representations. Classifiers trained with dynamic and static representations make misclassifications for different samples. We combine them first by forming a feature vector composed of dynamic and static repr...

this paper, the GLVQ [28] algorithm is applied to design

Abstract not found

Recognition of connected handwritten digit strings is a challenging task due mainly to two problems: poor character segmentation and unreliable isolated character recognition. In this paper, we first present a rational B-spline representation of digit templates based on Pixel-to-Boundary Distance (PBD) maps. We then present a neural network approach to extract B-spline PBD templates and an evolutionary algorithm to optimize these templates. In total, 1,000 templates (100 templates for each of 10 classes) were extracted from and optimized on 10,426 training samples from the NIST Special Database 3. By using these templates, a nearest neighbor classifier can successfully reject 90.7 percent of nondigit patterns while achieving a 96.4 percent correct classification of isolated test digits. When our classifier is applied to the recognition of 4,958 connected handwritten digit strings (4,555 2-digit, 355 3-digit, and 48 4-digit strings) from the NIST Special Database 3 with a dynamic programming approach, it has a correct classification rate of 82.4 percent with a rejection rate of as low as 0.85 percent. Our classifier compares favorably in terms of correct classification rate and robustness with other classifiers that are tested.