Key words: robust sparse hyperplanes; second-order cone program; linear programming; breast cancer; molecular profiling; two-class high-dimensional data

Abstract. A sharp lower bound on the probability of a set defined by quadratic inequalities, given the first two moments of the distribution, can be efficiently computed using convex optimization. This result generalizes Chebyshev’s inequality for scalar random variables. Two semidefinite programming formulations are presented, with a constructive proof based on convex optimization duality and elementary linear algebra. Key words. Semidefinite programming, convex optimization, duality theory, Chebyshev inequalities, moment problems. AMS subject classifications. 90C22, 90C25, 60-08.

The Minimax Probability Machine (MPM) constructs a classifier, which provides a worst-case bound on the probability of misclassification of future data points based on reliable estimates of means and covariance matrices of the classes from the training data points, and achieves the comparative performance with a state-of-the-art classifier, the Support Vector Machine. In this paper, we eliminate the assumption of the unbiased weight for each class in the MPM and develop a critical extension, named Biased Minimax Probability Machine (BMPM), to deal with biased classification tasks, especially in the medical diagnostic applications. We outline the theoretical derivatives of the BMPM. Moreover, we demonstrate that this model can be transformed into a concave-convex Fractional Programming (FP) problem or a pseudoconcave problem. After illustrating our model with a synthetic dataset and applying it to the real-world medical diagnosis datasets, we obtain encouraging and promising experimental results.

This paper addresses the issue of feature selection for linear classifiers given the moments of the class conditional densities. The problem is posed as finding a minimal set of features such that the resulting classifier has a low misclassification error. Using a bound on the misclassification error involving the mean and covariance of class conditional densities and minimizing an L1 norm as an approximate criterion for feature selection, a second order programming formulation is derived. To handle errors in estimation of mean and covariances, a tractable robust formulation is also discussed. In a slightly different setting the Fisher discriminant is derived. Feature selection for Fisher discriminant is also discussed. Experimental results on synthetic data sets and on real life microarray data show that the proposed formulations are competitive with the state of the art linear programming formulation. 1.

Abstract not found

This paper proposes a computationally efficient class of nonparametric binary classification algorithms that generate nonlinear separating boundaries, with minimal tuning of learning parameters. We avoid the computational pitfalls of using extensive cross validation for model selection. For example, in Support Vector Machines (SVMs) [6], both the choice of kernels and corresponding kernel parameters is based on extensive cross validation experiments, making generating good SVM models computationally very difficult. Other algorithms, such as Minimax Probability Machine Classification (MPMC) [5], Neural Networks, and even ensemble methods such as Boosting, can suffer from the same computational pitfalls. The Minimax Probability Machine for Classification (MPMC), due to Lanckriet et al. [5], is a recent algorithm that has this characteristic. Given the means and covariance matrices of two classes, MPMC calculates a hyperplane that separates the data by minimizing the maximum probability of misclassification. As such, it generates both a classification and a bound on the expected error for future data. In the same paper, the MPMC is also extended to non-linear separating

The Minimax Probability Machine Classification (MPMC) framework [Lanckriet et al., 2002] builds classifiers by minimizing the maximum probability of misclassification, and gives direct estimates of the probabilistic accuracy bound# . The only assumptions that MPMC makes is that good estimates of means and covariance matrixes of the classes exist.

We consider the problem of choosing a linear classifier that minimizes misclassification probabilities in two-class classification, which is a bi-criterion problem, involving a trade-off between two objectives. We assume that the class-conditional distributions are Gaussian. This assumption makes it computationally tractable to find Pareto optimal linear classifiers whose classification capabilities are inferior to no other linear ones. The main purpose of this paper is to establish several robustness properties of those classifiers with respect to variations and uncertainties in the distributions. We also extend the results to kernel-based classification. Finally, we show how to carry out trade-off analysis empirically with a finite number of given labeled data. 1.

I would like to express sincere gratitude and thanks to my adviser, Dr. Chiranjib Bhat-tacharyya. With his interesting thoughts and ideas, inspiring ideals and friendly nature, he made sure I was filled with enthusiasm and interest to do research all through my PhD. He was always approachable and spent ample time with me and all my lab mem-bers for discussions, though he had a busy schedule. I also thank Prof. M. N. Murty, Dr. Samy Bengio (Google Labs, USA) and Prof. Aharon Ben-Tal (Technion, Israel) for their help and co-operation. I am greatly in debt to my parents, wife and other family members for supporting and encouraging me all through the PhD years. I thank all my lab members and friends, especially Karthik Raman, Sourangshu, Rashmin, Krishnan and Sivaramakrishnan, for their useful discussions and comments. I thank the Department of Science and Technology, India, for supporting me finan-cially during the PhD work. I would also like to take this opportunity to thank all the people who directly and indirectly helped in finishing my thesis. i Publications based on this Thesis

Abstract — Semi-supervised kernel learning is attracting increasing research interests recently. It works by learning an embedding of data from the input space to a Hilbert space using both labeled data and unlabeled data, and then searching for relations among the embedded data points. One of the most well-known semi-supervised kernel learning approaches is the spectral kernel learning methodology which usually tunes the spectral empirically or through optimizing some generalized performance measures. However, the kernel designing process does not involve the bias of a kernel-based learning algorithm, the deduced kernel matrix cannot necessarily facilitate a specific learning algorithm. To supplement the spectral kernel learning methods, this paper proposes a novel approach, which not only learns a kernel matrix by maximizing another generalized performance measure, the margin between two classes of data, but also leads directly to a convex optimization method for learning the margin parameters in support vector machines. Moreover, experimental results demonstrate that our proposed spectral kernel learning method achieves promising results against other spectral kernel learning methods. I.