Abstract not found

Boosting algorithms like AdaBoost and Arc-GV are iterative strategies to minimize a constrained objective function, equivalent to Barrier algorithms.

We examine methods for constructing regression ensembles based on a linear program (LP). The ensemble regression function consists of linear combina- tions of base hypotheses generated by some boosting-type base learning algorithm. Unlike the classification case, for regression the set of possible hypotheses producible by the base learning algorithm may be infinite. We explicitly tackle the issue of how to define and solve ensemble regression when the hypothesis space is infinite. Our approach is based on a semi-infinite linear program that has an infinite number of constraints and a finite number of variables. We show that the regression problem is well posed for infinite hypothesis spaces in both the primal and dual spaces. Most importantly, we prove there exists an optimal solution to the infinite hypothesisspace problem consisting of a finite number of hypothesis. We propose two algorithms for solving the infinite and finite hypothesis problems. One uses a column generation simplex-type algorithm and the other adopts an exponential barrier approach. Furthermore, we give sufficient conditions for the base learning algorithm and the hypothesis set to be used for infinite regression ensembles. Computational resultsshow that these methods are extremely promising.

In this paper, we discuss a spectral quadratic-SDP method for the iterative resolution of fixed-order H2 and H ∞ design problems. These problems can be cast as regular SDP programs with additional nonlinear equality constraints. When the inequalities are absorbed into a Lagrangian function the problem reduces to solving a sequence of SDPs with quadratic objective function for which a spectral SDP method has been developed. Along with a description of the spectral SDP method used to solve the tangent subproblems, we report a number of computational results for validation purposes.

We show via an equivalence of mathematical programs that a Support Vector (SV) algorithm can be translated into an equivalent boosting-like algorithm and vice versa. We exemplify this translation procedure for a new algorithm one-class Leveraging starting from the one-class Support Vector Machines (1SVM) . This is a first step towards unsupervised learning in a Boosting framework.

We study nonsmooth unconstrained optimization problem, which includes sum of pairwise maxima of smooth functions. Minimum l 1-norm approximation is a particular case of this problem. Combining ideas Lagrange multipliers with smooth approximation of max-type function, we obtain a new kind of nonquadratic augmented Lagrangian. Our approach does not require arti cial variables, and preserves sparse structure of Hessian in many practical cases. We present the corresponding method of multipliers, and its convergence analysis for a dual counterpart, resulting in a proximal point maximization algorithm. The practical eciency of the algorithm is supported by computational results for large-scale problems, arising in structural optimization.

We introduce a new "sparse" nonlinear transformation of the SVM feature space, which permits the use of efficient optimization techniques for finding separating hyperplane. Corresponding quadratic program can be solved in the primal formulation, so that complexity of solution grows only linearly with the number of training examples. The Penalty/Barrier Multiplier method (a kind of non-quadratic Augmented Lagrangian) was successfully applied for the above mentioned problem. It provides a very high accuracy of 10 - 12 digits under moderate value of the penalty parameter. 10 - 20 Newton steps are normally sufficient to solve a problem. 1 Problem Formulation and the Main Approach In this paper we deal with so called Support Vector Machine [7] for pattern recognition via separation by optimal hyperplane in extended feature space. Consider a pattern recognition problem in the following setting. Let f : R n ! R separate two classes, represented by sets A; B 2 R n : f(x) = 1; x 2 A 1...

Abstract. We study a relative optimization framework for quasi-maximum likelihood blind source separation and relative Newton method as its particular instance. The structure of the Hessian allows its fast approximate inversion. In the second part we present Smoothing Method of Multipliers (SMOM) for minimization of sum of pairwise maxima of smooth functions, in particular sum of absolute value terms. Incorporating Lagrange multiplier into a smooth approximation of max-type function, we obtain an extended notion of non-quadratic augmented Lagrangian. Our approach does not require artificial variables, and preserves the sparse structure of Hessian. Convergence of the method is further accelerated by the Frozen Hessian strategy. We demonstrate efficiency of this approach on an example of blind separation of sparse sources. The non-linearity in this case is based on the absolute value function, which provides super-efficient source separation. In this chapter we study quasi-maximum likelihood blind source separation (quasi-ML BSS) [1,2] in batch mode, without orthogonality constraint. This

Abstract In this paper, we discuss a spectral quadratic-SDP method for the iterative resolution of fixed-order H2 and H1 design problems. These problems can be cast as regular SDP programs with additional nonlinear equality constraints. When the inequalities are absorbed into a Lagrangian function the problem reduces to solving a sequence of SDPs with quadratic objective function for which a spectral SDP method has been developed. Besides a description of the spectral SDP method used to solve the tangent subproblems, we report a number of computational results for validation purposes.

output feedback control design with specSDP, a code