Results 1  10
of
77
Support vector machines: Training and applications
 A.I. MEMO 1602, MIT A. I. LAB
, 1997
"... The Support Vector Machine (SVM) is a new and very promising classification technique developed by Vapnik and his group at AT&T Bell Laboratories [3, 6, 8, 24]. This new learning algorithm can be seen as an alternative training technique for Polynomial, Radial Basis Function and MultiLayer Perceptr ..."
Abstract

Cited by 177 (3 self)
 Add to MetaCart
The Support Vector Machine (SVM) is a new and very promising classification technique developed by Vapnik and his group at AT&T Bell Laboratories [3, 6, 8, 24]. This new learning algorithm can be seen as an alternative training technique for Polynomial, Radial Basis Function and MultiLayer Perceptron classifiers. The main idea behind the technique is to separate the classes with a surface that maximizes the margin between them. An interesting property of this approach is that it is an approximate implementation of the Structural Risk Minimization (SRM) induction principle [23]. The derivation of Support Vector Machines, its relationship with SRM, and its geometrical insight, are discussed in this paper. Since Structural Risk Minimization is an inductive principle that aims at minimizing a bound on the generalization error of a model, rather than minimizing the Mean Square Error over the data set (as Empirical Risk Minimization methods do), training a SVM to obtain the maximum margin classi er requires a different objective function. This objective function is then optimized by solving a largescale quadratic programming problem with linear and box constraints. The problem is considered challenging, because the quadratic form is completely dense, so the memory
Newton's Method For Large BoundConstrained Optimization Problems
 SIAM JOURNAL ON OPTIMIZATION
, 1998
"... We analyze a trust region version of Newton's method for boundconstrained problems. Our approach relies on the geometry of the feasible set, not on the particular representation in terms of constraints. The convergence theory holds for linearlyconstrained problems, and yields global and superlinea ..."
Abstract

Cited by 74 (4 self)
 Add to MetaCart
We analyze a trust region version of Newton's method for boundconstrained problems. Our approach relies on the geometry of the feasible set, not on the particular representation in terms of constraints. The convergence theory holds for linearlyconstrained problems, and yields global and superlinear convergence without assuming neither strict complementarity nor linear independence of the active constraints. We also show that the convergence theory leads to an efficient implementation for large boundconstrained problems.
Computational methods for sparse solution of linear inverse problems
, 2009
"... The goal of sparse approximation problems is to represent a target signal approximately as a linear combination of a few elementary signals drawn from a fixed collection. This paper surveys the major practical algorithms for sparse approximation. Specific attention is paid to computational issues, ..."
Abstract

Cited by 60 (0 self)
 Add to MetaCart
The goal of sparse approximation problems is to represent a target signal approximately as a linear combination of a few elementary signals drawn from a fixed collection. This paper surveys the major practical algorithms for sparse approximation. Specific attention is paid to computational issues, to the circumstances in which individual methods tend to perform well, and to the theoretical guarantees available. Many fundamental questions in electrical engineering, statistics, and applied mathematics can be posed as sparse approximation problems, making these algorithms versatile and relevant to a wealth of applications.
Twometric projection methods for constrained optimization
 SIAM Journal on Control and Optimization
, 1984
"... Abstract. This paper is concerned with the problem min {f(x)lx X} where X is a convex subset of a linear space H, and f is a smooth realvalued function on H. We propose the class of methods Xk+l P(xk akgk), where P denotes projection on X with respect to a Hilbert space norm II ’ [I, gk denotes th ..."
Abstract

Cited by 42 (2 self)
 Add to MetaCart
Abstract. This paper is concerned with the problem min {f(x)lx X} where X is a convex subset of a linear space H, and f is a smooth realvalued function on H. We propose the class of methods Xk+l P(xk akgk), where P denotes projection on X with respect to a Hilbert space norm II ’ [I, gk denotes the Frechet derivative of f at xk with respect to another Hilbert space norm I " on H, and ak is a positive scalar stepsize. We thus remove an important restriction in the original proposal of Goldstein and Levitin and Pofjak [2], where the norms arid II ’ II must be the same. It is therefore possible to match the norm II " with the structure of X so that the projection operation is simplified while at the same time reserving the option to choose 1. Ik on the basis of approximations to the Hessian of f so as to attain a typically superlinear rate of convergence. The resulting methods are particularly attractive for largescale problems with specially structured constraint sets such as optimal control and nonlinear multicommodity network flow problems. The latter class of problems is discussed in some detail. Key words, constrained optimization, gradient projection, convergence analysis, multicommodity flow problems, largescale optimization
Conditional random fields for activity recognition
 In Proceedings of the Sixth International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS 2007
, 2007
"... of any sponsoring institution, the U.S. government or any other entity. ..."
Abstract

Cited by 38 (0 self)
 Add to MetaCart
of any sponsoring institution, the U.S. government or any other entity.
LBFGSB  Fortran Subroutines for LargeScale Bound Constrained Optimization
, 1994
"... LBFGSB is a limited memory algorithm for solving large nonlinear optimization problems subject to simple bounds on the variables. It is intended for problems in which information on the Hessian matrix is di cult to obtain, or for large dense problems. LBFGSB can also be used for unconstrained pr ..."
Abstract

Cited by 38 (2 self)
 Add to MetaCart
LBFGSB is a limited memory algorithm for solving large nonlinear optimization problems subject to simple bounds on the variables. It is intended for problems in which information on the Hessian matrix is di cult to obtain, or for large dense problems. LBFGSB can also be used for unconstrained problems, and in this case performs similarly to its predecessor, algorithm LBFGS (Harwell routine VA15). The algorithm is implemented in Fortran 77.
Simplicial Decomposition with Disaggregated Representation for the Traffic Assignment Problem
 Transportation Science
, 1991
"... The class of simplicial decomposition (SD) schemes have shown to provide efficient tools for nonlinear network flows. When applied to the traffic assignment problem, shortest route subproblems are solved in order to generate extreme points of the polyhedron of feasible flows, and, alternately, maste ..."
Abstract

Cited by 32 (20 self)
 Add to MetaCart
The class of simplicial decomposition (SD) schemes have shown to provide efficient tools for nonlinear network flows. When applied to the traffic assignment problem, shortest route subproblems are solved in order to generate extreme points of the polyhedron of feasible flows, and, alternately, master problems are solved over the convex hull of the generated extreme points. We review the development of simplicial decomposition and the closely related column generation methods for the traffic assignment problem; we then present a modified, disaggregated, representation of feasible solutions in SD algorithms for convex problems over Cartesian product sets, with application to the symmetric traffic assignment problem. The new algorithm, which is referred to as disaggregate simplicial decomposition (DSD), is given along with a specialized solution method for the disaggregate master problem. Numerical results for several well known test problems and a new one are presented. These experimenta...
Interior Point Methods For Optimal Control Of DiscreteTime Systems
 Journal of Optimization Theory and Applications
, 1993
"... . We show that recently developed interior point methods for quadratic programming and linear complementarity problems can be put to use in solving discretetime optimal control problems, with general pointwise constraints on states and controls. We describe interior point algorithms for a discrete ..."
Abstract

Cited by 31 (5 self)
 Add to MetaCart
. We show that recently developed interior point methods for quadratic programming and linear complementarity problems can be put to use in solving discretetime optimal control problems, with general pointwise constraints on states and controls. We describe interior point algorithms for a discrete time linearquadratic regulator problem with mixed state/control constraints, and show how it can be efficiently incorporated into an inexact sequential quadratic programming algorithm for nonlinear problems. The key to the efficiency of the interiorpoint method is the narrowbanded structure of the coefficient matrix which is factorized at each iteration. Key words. interior point algorithms, optimal control, banded linear systems. 1. Introduction. The problem of optimal control of an initial value ordinary differential equation, with Bolza objectives and mixed constraints, is min x;u Z T 0 L(x(t); u(t); t) dt + OE f (x(T )); x(t) = f(x(t); u(t); t); x(0) = x init ; (1.1) g(x(t); u(...
On The Maximization Of A Concave Quadratic Function With Box Constraints
, 1994
"... . We introduce a new method for maximizing a concave quadratic function with bounds on the variables. The new algorithm combines conjugate gradients with gradient projection techniques, as the algorithm of Mor'e and Toraldo (SIAM J. on Optimization 1, pp. 93113) and other wellknown methods do. A n ..."
Abstract

Cited by 31 (11 self)
 Add to MetaCart
. We introduce a new method for maximizing a concave quadratic function with bounds on the variables. The new algorithm combines conjugate gradients with gradient projection techniques, as the algorithm of Mor'e and Toraldo (SIAM J. on Optimization 1, pp. 93113) and other wellknown methods do. A new strategy for the decision of leaving the current face is introduced, that makes it possible to obtain finite convergence even for a singular Hessian and in the presence of dual degeneracy. We present numerical experiments. November 4, 1992 0() Work supported by FAPESP (Grant 90/3724/6), FINEP, CNPq and FAEPUNICAMP. This paper appeared in SIAM Journal on Optimization 4 (1994) 177192 1. Introduction. In this paper, we consider the problem of maximizing a concave quadratic function subject to bounds on the variables. This problem (or its equivalent one: minimizing a convex quadratic function on a box) appears frequently in applications, for instance in finite difference discretization ...
Fast newtontype methods for the least squares nonnegative matrix approximation problem
 Statistical Analysis and Data Mining
, 2008
"... Nonnegative Matrix Approximation is an effective matrix decomposition technique that has proven to be useful for a wide variety of applications ranging from document analysis and image processing to bioinformatics. There exist a few algorithms for nonnegative matrix approximation (NNMA), for example ..."
Abstract

Cited by 31 (5 self)
 Add to MetaCart
Nonnegative Matrix Approximation is an effective matrix decomposition technique that has proven to be useful for a wide variety of applications ranging from document analysis and image processing to bioinformatics. There exist a few algorithms for nonnegative matrix approximation (NNMA), for example, Lee & Seung’s multiplicative updates, alternating least squares, and certain gradient descent based procedures. All of these procedures suffer from either slow convergence, numerical instabilities, or at worst, theoretical unsoundness. In this paper we present new and improved algorithms for the leastsquares NNMA problem, which are not only theoretically wellfounded, but also overcome many of the deficiencies of other methods. In particular, we use nondiagonal gradient scaling to obtain rapid convergence. Our methods provide numerical results superior to both Lee & Seung’s method as well to the alternating least squares (ALS) heuristic, which is known to work well in some situations but has no theoretical guarantees (Berry et al. 2006). Our approach extends naturally to include regularization and boxconstraints, without sacrificing convergence guarantees. We present experimental results on both synthetic and realworld datasets to demonstrate the superiority of our methods, in terms of better approximations as well as efficiency.