We review the literature on semi-supervised learning, which is an area in machine learning and more generally, artificial intelligence. There has been a whole spectrum of interesting ideas on how to learn from both labeled and unlabeled data, i.e. semi-supervised learning. This document is a chapter excerpt from the author’s doctoral thesis (Zhu, 2005). However the author plans to update the online version frequently to incorporate the latest development in the field. Please obtain the latest version at http://www.cs.wisc.edu/~jerryzhu/pub/ssl_survey.pdf

In this paper, on the one hand, we aim to give a review on literature dealing with the problem of supervised learning aided by additional unlabeled data. On the other hand, being a part of the author's first year PhD report, the paper serves as a frame to bundle related work by the author as well as numerous suggestions for potential future work. Therefore, this work contains more speculative and partly subjective material than the reader might expect from a literature review. We give a rigorous definition of the problem and relate it to supervised and unsupervised learning. The crucial role of prior knowledge is put forward, and we discuss the important notion of input-dependent regularization. We postulate a number of baseline methods, being algorithms or algorithmic schemes which can more or less straightforwardly be applied to the problem, without the need for genuinely new concepts. However, some of them might serve as basis for a genuine method. In the literature revi...

This paper presents two new formulations of multiple-instance learning as a maximum margin problem. The proposed extensions of the Support Vector Machine (SVM) learning approach lead to mixed integer quadratic programs that can be solved heuristically. Our generalization of SVMs makes a state-of-the-art classification technique, including non-linear classification via kernels, available to an area that up to now has been largely dominated by special purpose methods. We present experimental results on a pharmaceutical data set and on applications in automated image indexing and document categorization. 1

no-Perez 1998; Zhang & Goldman 2002)) have focused on specially tailored machine learning algorithms that do not compare favorably in the limiting case of bags of size 1 (the standard classification setting). A notable exception is (Ramon & Raedt 2000). Generalized Support Vector Machines We propose to generalize Support Vector Machines (SVMs) (Vapnik 1998) to take into account weak labeling information of the type found in MIL. SVMs are based on the theory of linear classifiers, more precisely the idea of the maximum margin hyperplane.For linearly separable data, the maximum margin hyperplane is Copyright c 2002, American Association for Artificial Intelligence (www.aaai.org). All rights reserved. defined by parameters w # ,b # with (w # ,b # ) = arg max (w,b),#w#=1 min i y i (#w, x i + b) (1) The minimum # # =min i # i is called the (geometric) margin and the patterns x i with # i = # # are called support vectors. The so-called soft-margin generalization of SVMs w

Learning from ambiguous training data is highly relevant in many applications. We present a new learning algorithm for classification problems where labels are associated with sets of pattern instead of individual patterns. This encompasses multiple instance learning as a special case. Our approach is based on a generalization of linear programming boosting and uses results from disjunctive programming to generate successively stronger linear relaxations of a discrete non-convex problem. 1

Abstract. The successful application of Support Vector Machines (SVMs), kernel methods and other statistical machine learning methods requires selection of model parameters based on estimates of the generalization error. This paper presents a novel approach to systematic model selection through bilevel optimization. We show how modelling tasks for widely used machine learning methods can be formulated as bilevel optimization problems and describe how the approach can address a broad range of tasks—among which are parameter, feature and kernel selection In addition, we also discuss the challenges in implementing these approaches and enumerate opportunities for future work in this emerging research area. 1.

Polycategorical classification deals with the task of solving multiple interdependent classification problems. The key challenge is to systematically exploit possible dependencies among the labels to improve on the standard approach of solving each classification problem independently.

Amongst the wealth of applications that linear programming (LP) and quadratic programming (QP) possess, we highlight in this article some of them which we consider interesting enough to motivate researching in or teaching of this subject. We also stress the importance of solving in an efficient way a sequence of related problems from several areas, in particular when solving optimization problems using active-set methods. Key words: linear programming, quadratic programming, active-set methods, numerical linear algebra, applications 1 LP and QP engineering applications Roughly speaking, constrained optimization problems are those in which the best value of some given function has to be found. Thus, a generic constrained optimization problem involves a set of variables from among the values of which is the solution of the problem, some constraints that restrict the acceptable values of such variables, and an objective function to be optimized (minimized or maximized) that depends on the variables. Constraints may be of two types: an equality constraint specifies that a certain function of the variables must be constant, whereas an inequality constraint specifies that a certain function of the variables must be greater than or equal to (or less than or equal to) a constant. On the other hand, a linear programming problem or linear program (LP) is a constrained optimization problem in which the objective function and all the constraint functions are linear; in the same case as above, a quadratic programming problem or quadratic program (QP) is given when the objective function is a quadratic form.

Abstract not found