A fundamental problem in computer vision and pattern recognition is to determine where and, most importantly, why a given technique is applicable. This is not only necessary because it helps us decide which techniques to apply at each given time. Knowing why current algorithms cannot be applied, facilitates the design of new algorithms robust to such problems. In this paper, we report on a theoretical study that demonstrates where and why generalized eigen-based linear equations do not work. In particular, we show that when the smallest angle between the i th eigenvector given by the metric to be maximized and the first i eigenvectors given by the metric to be minimized is close to zero, our results are not guaranteed to be correct. Several properties of such models are also presented. For illustration, we concentrate on the classical applications of classification and feature extraction. We also show how we can use our findings to design more robust algorithms. We conclude with a discussion on the broader impacts of our results. Index terms: feature extraction, generalized eigenvalue decomposition, performance evaluation, classifiers, pattern recognition. 1

In current face detection, mostly often used features are selected from a large set (e.g. Haar wavelets). Generally Haar wavelets only represent the local geometric feature. When applying those features to profile faces and eyes with irregular geometric patterns, the classifier accuracy is low in the later training stages, only near 50%. In this paper, instead of brute-force searching the large feature set, we propose to statistically learn the discriminant features for object detection. Besides applying Fisher discriminant analysis(FDA) in AdaBoost, we further propose the recursive nonparametric discriminant analysis (RNDA) to handle more general cases. Those discriminant analysis features are not constrained with geometric shape and can provide better accuracy. The compact size of feature set allows to select a near optimal subset of features and construct the probabilistic classifiers by greedy searching. The proposed methods are applied to multi-view face and eye detection and achieve good accuracy. 1

We present an algorithm which provides the one-dimensional subspace where the Bayes error is minimized for the C class problem with homoscedastic Gaussian distributions. Our main result shows that the set of possible one-dimensional spaces v, for which the order of the projected class means is identical, defines a convex region with associated convex Bayes error function g(v). This allows for the minimization of the error function using standard convex optimization algorithms. Our algorithm is then extended to the minimization of the Bayes error in the more general case of heteroscedastic distributions. This is done by means of an appropriate kernel mapping function. This result is further extended to obtain the d-dimensional solution for any given d, by iteratively applying our algorithm to the null space of the (d − 1)-dimensional solution. We also show how this result can be used to improve upon the outcomes provided by existing algorithms, and derive a low-computational cost, linear approximation. Extensive experimental validations are provided to demonstrate the use of these algorithms in classification, data analysis and visualization. Index terms: Linear discriminant analysis, feature extraction, Bayes optimal, convex optimization, pattern recognition, data mining, data visualization. 1

vii CHAPTER 1:

This material is posted here with permission of the IEEE. Internal or personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution must be obtained from the IEEE by sending a blank email message to

Dimensionality reduction is widely acceptes as an analysis and modeling tool to deal with high-dimensional spaces, although researches from different disciplines have different interpretations of what properties should be preserved in the reduction process. We study the problem of linear dimension reduction for classification, with a focus on sufficient dimension reduction, i.e., inducing subspaces without loss of discriminative information. Decision boundary analysis (DBA), originally proposed by Lee & Landgrebe (1993), can directly find the smallest subspace with such property. However, existing DBA implementations are computationally expensive and sensitive to sample size. In this paper, we first formulate the problem of sufficient dimension reduction for classification in parallel terms as for regression. Disclosures of these connections lead to several meaningful observations. Then we present a novel space reduction algorithm that combines SVM and DBA, thus inheriting several appealing properties from kernel machines such as good generalization, weak assumption, and efficient computation. In addition, the proposed method provides a natural way to reduce the complexity, and even improve the accuracy, of SVM itself. We demonstrate its superiority by comparative experiments on one simulated and four real-world benchmark datasets.

Abstract: Traffic sign classification is a challenging problem in Computer Vision due to the high variability of sign appearance in uncontrolled environments. Lack of visibility, illumination changes, and partial occlusions are just a few problems. In this paper, we introduce a classification technique for traffic signs recognition by means of Error Correcting Output Codes. Recently, new proposals of coding and decoding strategies for the Error Correcting Output Codes framework have been shown to be very effective in front of multiclass problems. We review the state-of-the-art ECOC strategies and combinations of problem-dependent coding designs and decoding techniques. We apply these approaches to the Mobile Mapping problem. We detect the sign regions by means of Adaboost. The Adaboost in an attentional cascade with the extended set of Haar-like features estimated on the integral shows great performance at the detection step. Then, a spatial normalization using the Hough transform and the fast radial symmetry is done. The model fitting improves the final classification performance by normalizing the sign content. Finally, we classify a wide set of traffic signs types, obtaining high success in adverse conditions. 1

This paper formulates the problem of object categorization in the discriminant analysis framework focusing on transforming visual feature data so as to make it conform to the compactness hypothesis in order to improve categorization accuracy. The sought transformation, in turn, is found as a solution to an optimization problem formulated in terms of inter-observation distances only, using the technique of iterative majorization. The proposed approach is suitable for both binary and multiple-class categorization problems, and can be applied as a dimensionality reduction technique. In the latter case, the number of discriminative features is determined automatically since the process of feature extraction is fully embedded in the optimization procedure. Performance tests validate our method on a number of benchmark data sets from the UCI repository, while the experiments in the application of visual object and contentbased image categorization demonstrate very competitive results, asserting the method's capability of producing semantically relevant matches that share the same or synonymous vocabulary with the query category and allowing multiple pertinent category assignment.

The most classical way of attempting to solve the vision-guided navigation problem for autonomous robots, corresponds to the use of 3D geometrical descriptions of the scene; what is known as model-based approaches. However, these approaches do not facilitate the user's task because they require a geometrically precise models of the 3D environment be given by the user. In this paper, we propose the use of "annotations" posted on some type of blackboard or "descriptive" map to facilitate this user-robot interaction. We will showhow using this technique the user's command can be as easier as: #go to label 5". To build such a mechanism, new approaches for vision-guided mobile robot navigation havetobe found. We will showhow this can be achieve by means of mixture models within an appearance-based paradigm. Mixture models have proven to be more useful, in practice, than other pattern recognition methods such as PCA (Principal Component Analysis) or DA (Discriminant Analysis), because they a...

Distance metric is widely used in similarity estimation. In this paper we find that the most popular Euclidean and Manhattan distance may not be suitable for all data distribution. A general guideline to establish the relation between a distribution model and its corresponding similarity estimation is proposed. Based on Maximum Likelihood theory, we propose new distance metrics, such as harmonic distance and geometric distance. Because the feature elements may be from heterogeneous sources and usually have different influence on similarity estimation, it is inappropriate to model the distribution as isotropic. We propose a novel boosted distance metric that not only finds the best distance metric that fits the distribution of the underlying elements but also selects the most important feature elements with respect to similarity. The boosted distance metric is tested on fifteen benchmark data sets from the UCI repository and two image retrieval applications. In all the experiments, robust results are obtained based on the proposed methods. 1.