## Joint Induction of Shape Features and Tree Classifiers (1997)

### Cached

### Download Links

- [www.wisdom.weizmann.ac.il]
- [galton.uchicago.edu]
- [www.math.umass.edu]
- [www.cs.berkeley.edu]
- [www.ics.uci.edu]
- [www.eecs.berkeley.edu]
- DBLP

### Other Repositories/Bibliography

Venue: | IEEE Trans. PAMI |

Citations: | 75 - 6 self |

### BibTeX

@ARTICLE{Geman97jointinduction,

author = {Donald Geman and Yali Amit and Ken Wilder},

title = {Joint Induction of Shape Features and Tree Classifiers},

journal = {IEEE Trans. PAMI},

year = {1997},

volume = {19}

}

### Years of Citing Articles

### OpenURL

### Abstract

We introduce a very large family of binary features for two-dimensional shapes. The salient ones for separating particular shapes are determined by inductive learning during the construction of classi cation trees. There is a feature for every possible geometric arrangement of local topographic codes. The arrangements express coarse constraints on relative angles and distances among the code locations and are nearly invariant to substantial a ne and non-linear deformations. They are also partially ordered, which makes it possible to narrow the search for informative ones at each node of the tree. Di erent trees correspond to di erent aspects of shape. They are statistically weakly dependent due to randomization and are aggregated in a simple way. Adapting the algorithm to a shape family is then fully automatic once training samples are provided. As an illustration, we classify handwritten digits from the NIST database � the error rate is:7%.

### Citations

3951 |
Classification and regression trees
- Breiman, Olsen, et al.
- 1984
(Show Context)
Citation Context ...t of slant-correcting and scaling. The best error rate we achieved with a single tree was about 7 percent. In contrast to standard recursive partitioning, we did not grow a deep tree and “prune back” =-=[28]-=-, which is an approach designed to avoid “overfitting.” In the context of multiple trees this problem appears to be of less importance. We stop splitting when the number of data points in the second l... |

2515 | Bagging predictors
- Breiman
- 1996
(Show Context)
Citation Context ...erage is (x) = 1 N NX n=1 n� (x) where N is the number of trees. The image x is classi ed by the class at the mode of (x). Alternative ways of producing and aggregating multiple trees are proposed in =-=[20]-=-,[21], 9sFigure 5: Arrangements found in an image at terminal nodes of six di erent trees. 10s[22],[23]. In [17] there is a statistical analysis of the dependence structure among the trees. Various re... |

1307 | A training algorithm for optimal margin classifiers
- Boser, Guyon, et al.
- 1992
(Show Context)
Citation Context ...) ([1]), and there is still no solution that matches human performance. Many approaches today are based on non-parametric statistical methods such as neural networks ([2],[3]), discriminant analysis (=-=[4]-=-,[5]), nearest-neighbor rules with di erent metrics ([6], [7],[8]), and classi cation trees ([9],[10]). Hybrid and multiple classi ers are also e ective ([11],[12]). In many cases the feature vector d... |

571 | G.: Solving multiclass learning problems via errorcorrecting output codes
- Dietterich, Bakiri
- 1995
(Show Context)
Citation Context ...at the mode of (x). Alternative ways of producing and aggregating multiple trees are proposed in [20],[21], 9sFigure 5: Arrangements found in an image at terminal nodes of six di erent trees. 10s[22],=-=[23]-=-. In [17] there is a statistical analysis of the dependence structure among the trees. Various rejection criteria can also be de ned in terms of . For example an image x is classi ed only if the value... |

316 | Decision combination in multiple classifier systems
- Ho, Hull, et al.
- 1994
(Show Context)
Citation Context ... networks [2], [3], discriminant analysis [4], [5], nearest-neighbor rules with different metrics [6], [7], [8], and classification trees [9], [10]. Hybrid and multiple classifiers are also effective =-=[11]-=-, [12]. In many cases the feature vector does not explicitly address “shape.” Our features are shape-based and bear a resemblance to the geometric invariants which have been proposed for recognizing r... |

230 |
Geometric Invariance in Computer Vision
- Mundy, Zisserman
- 1992
(Show Context)
Citation Context ...rmation, computing tangents and identifying distinguished points, such as those of high curvature or in ections. Invariant geometric relations are then determined among these special points� see e.g.,=-=[14]-=-,[15], [16]. Many authors report much better results with these and \structural features" than with standardized raw data. Our features also involve geometric relations among points, but not distingui... |

195 |
Classi - cation and regression trees
- Breiman, Friedman, et al.
- 1984
(Show Context)
Citation Context ...e result of slant-correcting and scaling. The best error rate we achieved with a single tree was about 7%. In contrast to standard recursive partitioning, we did not grow a deeptree and \prune back" (=-=[26]-=-), which is an approach designed to avoid \over- tting." In the context of multiple trees this problem appears to be of less importance. We stop splitting when the number of data points in the second ... |

187 | Handwritten digit recognition with a back-propagation network - LeCun, Boser, et al. - 1990 |

187 | Shape quantization and recognition with randomized trees
- Amit, Geman
- 1997
(Show Context)
Citation Context ... is part of the training process and there is no dedicated modeling. As a result, the method is entirely portable� for example, it has been applied to recognizing deformed LaTeX symbols (293 classes) =-=[17]-=- and rigid three-dimensional objects [18]. The classi er is constructed from multiple classi cation trees. Randomization prevents the same features from being chosen from tree to tree and guarantees w... |

170 |
A database for handwritten text recognition research
- Hull
- 1994
(Show Context)
Citation Context ... (\NIST Test Data 1") was considered quite di cult and recognition rates reported elsewhere, for instance on portions of the NIST training set (\NIST Special Database 3" [19]) or on the USPS test set =-=[24]-=-, are generally higher. For example, using a nearest-neighbor system, the study in [8] achieves 98:61% with 100� 000 training points on the NIST training database. The best reported rates seem to be t... |

131 | B.: Sparse discriminant analysis
- Clemmensen, Hastie, et al.
- 2011
(Show Context)
Citation Context ...1]), and there is still no solution that matches human performance. Many approaches today are based on non-parametric statistical methods such as neural networks ([2],[3]), discriminant analysis ([4],=-=[5]-=-), nearest-neighbor rules with di erent metrics ([6], [7],[8]), and classi cation trees ([9],[10]). Hybrid and multiple classi ers are also e ective ([11],[12]). In many cases the feature vector does ... |

131 |
A method of combining multiple experts for the recognition of unconstrained handwritten numerals
- Huang, Suen
- 1995
(Show Context)
Citation Context ...rks ([2],[3]), discriminant analysis ([4],[5]), nearest-neighbor rules with di erent metrics ([6], [7],[8]), and classi cation trees ([9],[10]). Hybrid and multiple classi ers are also e ective ([11],=-=[12]-=-). In many cases the feature vector does not explicitly address \shape." Our features are shape-based and bear a resemblance to the geometric invariants which have been proposed for recognizing rigid ... |

79 |
Boosting performance in neural networks
- Drucker, Schapire, et al.
- 1993
(Show Context)
Citation Context ... test set is shown in Figure 6 (top). The studies mentioned above utilize pre-processing, such as thinning, slant correction and size normalization. Many also utilize post-processing� for example, in =-=[25]-=- it is shown how to use additional training data to make a second classi er more or less dedicated to the mistakes and marginal decisions of the original classi er� see also [2]. This procedure (\boos... |

74 |
Recognizing Planar Objects Using Invariant Image Features
- Reiss
- 1993
(Show Context)
Citation Context ...t explicitly address \shape." Our features are shape-based and bear a resemblance to the geometric invariants which have been proposed for recognizing rigid shapes and three-dimensional objects. (See =-=[13]-=- for a review and bibliography.) Typically this involves extracting boundary information, computing tangents and identifying distinguished points, such as those of high curvature or in ections. Invari... |

71 | Comparison of classifier methods: A case study in handwritten digit recognition - Bottou, Cortes, et al. - 1994 |

60 |
Multiple decision trees
- Kwok, Carter
- 1990
(Show Context)
Citation Context ...trees were reported in [21] and there is a statistical analysis of the dependence structure among the trees in [17]. Alternative ways of producing and aggregating multiple trees are proposed in [22], =-=[23]-=-, [24], [25], and [19]. Various rejection criteria can also be defined in terms of P . For example, an image x is classified only if the value at the mode of P16 x exceeds some threshold or exceeds so... |

40 |
The second census optical character recognition systems conference
- Geist, Wilkinson, et al.
- 1994
(Show Context)
Citation Context .... We experiment with isolated handwritten digits. O -line recognition has attracted enormous attention, including a competition sponsored by the National Institute of Standards and Technology (NIST) (=-=[1]-=-), and there is still no solution that matches human performance. Many approaches today are based on non-parametric statistical methods such as neural networks ([2],[3]), discriminant analysis ([4],[5... |

23 |
Analysis and design of a decision tree based on entropy reduction and its application to large character set recognition
- Wang, Suen
- 1984
(Show Context)
Citation Context ...sed on non-parametric statistical methods such as neural networks ([2],[3]), discriminant analysis ([4],[5]), nearest-neighbor rules with di erent metrics ([6], [7],[8]), and classi cation trees ([9],=-=[10]-=-). Hybrid and multiple classi ers are also e ective ([11],[12]). In many cases the feature vector does not explicitly address \shape." Our features are shape-based and bear a resemblance to the geomet... |

21 |
Learning Shape Classes
- Cho, Dunn
- 1994
(Show Context)
Citation Context ...mputing tangents and identifying distinguished points, such as those of high curvature or in ections. Invariant geometric relations are then determined among these special points� see e.g.,[14],[15], =-=[16]-=-. Many authors report much better results with these and \structural features" than with standardized raw data. Our features also involve geometric relations among points, but not distinguished points... |

21 |
Handwritten character classification using nearest neighbor in large database
- Smith, Bourgoin, et al.
- 1994
(Show Context)
Citation Context ...ormance. Many approaches today are based on nonparametric statistical methods such as neural networks [2], [3], discriminant analysis [4], [5], nearest-neighbor rules with different metrics [6], [7], =-=[8]-=-, and classification trees [9], [10]. Hybrid and multiple classifiers are also effective [11], [12]. In many cases the feature vector does not explicitly address “shape.” Our features are shape-based ... |

18 |
Decision Combination in Multiple Classi er Systems
- Ho, Hull, et al.
- 1994
(Show Context)
Citation Context ...networks ([2],[3]), discriminant analysis ([4],[5]), nearest-neighbor rules with di erent metrics ([6], [7],[8]), and classi cation trees ([9],[10]). Hybrid and multiple classi ers are also e ective (=-=[11]-=-,[12]). In many cases the feature vector does not explicitly address \shape." Our features are shape-based and bear a resemblance to the geometric invariants which have been proposed for recognizing r... |

14 |
Multiple binary decision tree classifiers
- Shlien
- 1990
(Show Context)
Citation Context ... are based on nonparametric statistical methods such as neural networks [2], [3], discriminant analysis [4], [5], nearest-neighbor rules with different metrics [6], [7], [8], and classification trees =-=[9]-=-, [10]. Hybrid and multiple classifiers are also effective [11], [12]. In many cases the feature vector does not explicitly address “shape.” Our features are shape-based and bear a resemblance to the ... |

13 |
A training algorithm for optimal g
- Boser, Guyon, et al.
- 1998
(Show Context)
Citation Context ...(NIST) [1], and there is still no solution that matches human performance. Many approaches today are based on nonparametric statistical methods such as neural networks [2], [3], discriminant analysis =-=[4]-=-, [5], nearest-neighbor rules with different metrics [6], [7], [8], and classification trees [9], [10]. Hybrid and multiple classifiers are also effective [11], [12]. In many cases the feature vector ... |

11 |
Memory-based character recognition using a transformation invariant metric
- SIMARD, CUN, et al.
- 1994
(Show Context)
Citation Context ...rformance. Many approaches today are based on non-parametric statistical methods such as neural networks ([2],[3]), discriminant analysis ([4],[5]), nearest-neighbor rules with di erent metrics ([6], =-=[7]-=-,[8]), and classi cation trees ([9],[10]). Hybrid and multiple classi ers are also e ective ([11],[12]). In many cases the feature vector does not explicitly address \shape." Our features are shape-ba... |

9 |
Optical character recognition by a neural network
- Sabourin, Mitiche
- 1992
(Show Context)
Citation Context ...on, computing tangents and identifying distinguished points, such as those of high curvature or in ections. Invariant geometric relations are then determined among these special points� see e.g.,[14],=-=[15]-=-, [16]. Many authors report much better results with these and \structural features" than with standardized raw data. Our features also involve geometric relations among points, but not distinguished ... |

6 |
Nist special database 3: Handwritten segmented characters
- Garris, Wilkinson
(Show Context)
Citation Context ...and the bias/variance tradeo . The purpose here is to introduce the features, outline the algorithm and experiment with handwritten digits. The training and test sets are taken from the NIST database =-=[19]-=-. In terms of speed and accuracy we achieve results which are comparable to the best of those reported elsewhere. 2 Tags The rst step in the algorithm involves assigning each pixel in the image one or... |

6 |
Decision tree algorithms for handwritten digit recognition
- Wilder
- 1998
(Show Context)
Citation Context ...cted from multiple classification trees. Randomization prevents the same features from being chosen from tree to tree and guarantees weak dependence. These statistical issues are analyzed in [17] and =-=[19]-=-, together with semiinvariance, generalization and the bias/variance tradeoff. The purpose here is to introduce the features, outline the algorithm and experiment with handwritten digits. The training... |

5 |
Averaging over decision stumps
- Oliver, Hand
- 1994
(Show Context)
Citation Context ...lass at the mode of (x). Alternative ways of producing and aggregating multiple trees are proposed in [20],[21], 9sFigure 5: Arrangements found in an image at terminal nodes of six di erent trees. 10s=-=[22]-=-,[23]. In [17] there is a statistical analysis of the dependence structure among the trees. Various rejection criteria can also be de ned in terms of . For example an image x is classi ed only if the ... |

4 |
Comparison of classi er methods: a case study in handwritten digit recognition
- Bottou, Cortes, et al.
- 1994
(Show Context)
Citation Context ...of Standards and Technology (NIST) ([1]), and there is still no solution that matches human performance. Many approaches today are based on non-parametric statistical methods such as neural networks (=-=[2]-=-,[3]), discriminant analysis ([4],[5]), nearest-neighbor rules with di erent metrics ([6], [7],[8]), and classi cation trees ([9],[10]). Hybrid and multiple classi ers are also e ective ([11],[12]). I... |

4 | Randomized inquiries about shape; an application to handwritten digit recognition
- Amit, Geman
- 1994
(Show Context)
Citation Context ...classify an image x in the test set, we simply compute P16 x and take the mode of this distribution as the estimated class. Our first experiments with such multiple, randomized trees were reported in =-=[21]-=- and there is a statistical analysis of the dependence structure among the trees in [17]. Alternative ways of producing and aggregating multiple trees are proposed in [22], [23], [24], [25], and [19].... |

3 |
3d object recognition from geometric queries
- Jedynak, Fleuret
- 1996
(Show Context)
Citation Context ...e is no dedicated modeling. As a result, the method is entirely portable� for example, it has been applied to recognizing deformed LaTeX symbols (293 classes) [17] and rigid three-dimensional objects =-=[18]-=-. The classi er is constructed from multiple classi cation trees. Randomization prevents the same features from being chosen from tree to tree and guarantees weak dependence. These statistical issues ... |

3 |
Massively-Parallel Handwritten Character Recognition Based on
- Kovics, Guerrieri
- 1995
(Show Context)
Citation Context ...human performance. Many approaches today are based on nonparametric statistical methods such as neural networks [2], [3], discriminant analysis [4], [5], nearest-neighbor rules with different metrics =-=[6]-=-, [7], [8], and classification trees [9], [10]. Hybrid and multiple classifiers are also effective [11], [12]. In many cases the feature vector does not explicitly address “shape.” Our features are sh... |

2 |
Multiple binary decision tree classi ers
- Shlien
- 1990
(Show Context)
Citation Context ...e based on non-parametric statistical methods such as neural networks ([2],[3]), discriminant analysis ([4],[5]), nearest-neighbor rules with di erent metrics ([6], [7],[8]), and classi cation trees (=-=[9]-=-,[10]). Hybrid and multiple classi ers are also e ective ([11],[12]). In many cases the feature vector does not explicitly address \shape." Our features are shape-based and bear a resemblance to the g... |

1 |
Massively- parallel handwritten character recognition based on the distance transform
- Kovacs-V, Guerrieri
- 1995
(Show Context)
Citation Context ...an performance. Many approaches today are based on non-parametric statistical methods such as neural networks ([2],[3]), discriminant analysis ([4],[5]), nearest-neighbor rules with di erent metrics (=-=[6]-=-, [7],[8]), and classi cation trees ([9],[10]). Hybrid and multiple classi ers are also e ective ([11],[12]). In many cases the feature vector does not explicitly address \shape." Our features are sha... |

1 |
Handwritten character classi - cation using nearest neighbor in large databases
- Smith, Bourgoin, et al.
- 1994
(Show Context)
Citation Context ...mance. Many approaches today are based on non-parametric statistical methods such as neural networks ([2],[3]), discriminant analysis ([4],[5]), nearest-neighbor rules with di erent metrics ([6], [7],=-=[8]-=-), and classi cation trees ([9],[10]). Hybrid and multiple classi ers are also e ective ([11],[12]). In many cases the feature vector does not explicitly address \shape." Our features are shape-based ... |

1 |
Multiple decision trees," in Uncertainty and Arti cial Intelligence
- Kwok, Carter
- 1990
(Show Context)
Citation Context ... is (x) = 1 N NX n=1 n� (x) where N is the number of trees. The image x is classi ed by the class at the mode of (x). Alternative ways of producing and aggregating multiple trees are proposed in [20],=-=[21]-=-, 9sFigure 5: Arrangements found in an image at terminal nodes of six di erent trees. 10s[22],[23]. In [17] there is a statistical analysis of the dependence structure among the trees. Various rejecti... |