Results 1  10
of
67
A tutorial on support vector machines for pattern recognition
 Data Mining and Knowledge Discovery
, 1998
"... The tutorial starts with an overview of the concepts of VC dimension and structural risk minimization. We then describe linear Support Vector Machines (SVMs) for separable and nonseparable data, working through a nontrivial example in detail. We describe a mechanical analogy, and discuss when SV ..."
Abstract

Cited by 3324 (12 self)
 Add to MetaCart
(Show Context)
The tutorial starts with an overview of the concepts of VC dimension and structural risk minimization. We then describe linear Support Vector Machines (SVMs) for separable and nonseparable data, working through a nontrivial example in detail. We describe a mechanical analogy, and discuss when SVM solutions are unique and when they are global. We describe how support vector training can be practically implemented, and discuss in detail the kernel mapping technique which is used to construct SVM solutions which are nonlinear in the data. We show how Support Vector machines can have very large (even infinite) VC dimension by computing the VC dimension for homogeneous polynomial and Gaussian radial basis function kernels. While very high VC dimension would normally bode ill for generalization performance, and while at present there exists no theory which shows that good generalization performance is guaranteed for SVMs, there are several arguments which support the observed high accuracy of SVMs, which we review. Results of some experiments which were inspired by these arguments are also presented. We give numerous examples and proofs of most of the key theorems. There is new material, and I hope that the reader will find that even old material is cast in a fresh light.
A tutorial on support vector regression
, 2004
"... In this tutorial we give an overview of the basic ideas underlying Support Vector (SV) machines for function estimation. Furthermore, we include a summary of currently used algorithms for training SV machines, covering both the quadratic (or convex) programming part and advanced methods for dealing ..."
Abstract

Cited by 828 (3 self)
 Add to MetaCart
In this tutorial we give an overview of the basic ideas underlying Support Vector (SV) machines for function estimation. Furthermore, we include a summary of currently used algorithms for training SV machines, covering both the quadratic (or convex) programming part and advanced methods for dealing with large datasets. Finally, we mention some modifications and extensions that have been applied to the standard SV algorithm, and discuss the aspect of regularization from a SV perspective.
Energylatency tradeoffs for data gathering in wireless sensor networks
 In IEEE Infocom
, 2004
"... Abstract — We study the problem of scheduling packet transmissions for data gathering in wireless sensor networks. The focus is to explore the energylatency tradeoffs in wireless communication using techniques such as modulation scaling. The data aggregation tree – a multiplesource singlesink com ..."
Abstract

Cited by 111 (5 self)
 Add to MetaCart
Abstract — We study the problem of scheduling packet transmissions for data gathering in wireless sensor networks. The focus is to explore the energylatency tradeoffs in wireless communication using techniques such as modulation scaling. The data aggregation tree – a multiplesource singlesink communication paradigm – is employed for abstracting the packet flow. We consider a realtime scenario where the data gathering must be performed within a specified latency constraint. We present algorithms to minimize the overall energy dissipation of the sensor nodes in the aggregation tree subject to the latency constraint. For the offline problem, we propose (a) a numerical algorithm for the optimal solution, and (b) a pseudopolynomial time approximation algorithm based on dynamic programming. We also discuss techniques for handling interference among the sensor nodes. Simulations have been conducted for both longrange communication and shortrange communication. The simulation results show that compared with the classic shutdown technique, between 20 % to 90 % energy savings can be achieved by our techniques, under different settings of several key system parameters. We also develop an online distributed protocol that relies only on the local information available at each sensor node within the aggregation tree. Simulation results show that between 15 % to 90 % energy conservation can be achieved by the online protocol. The adaptability of the protocol with respect to variations in the packet size and latency constraint is also demonstrated through several runtime scenarios. Index terms – System design, Mathematical optimization I.
Global Optimization of MixedInteger Nonlinear Programs: A Theoretical and Computational Study
 Mathematical Programming
, 2003
"... This work addresses the development of an efficient solution strategy for obtaining global optima of continuous, integer, and mixedinteger nonlinear programs. Towards this end, we develop novel relaxation schemes, range reduction tests, and branching strategies which we incorporate into the prototy ..."
Abstract

Cited by 76 (2 self)
 Add to MetaCart
This work addresses the development of an efficient solution strategy for obtaining global optima of continuous, integer, and mixedinteger nonlinear programs. Towards this end, we develop novel relaxation schemes, range reduction tests, and branching strategies which we incorporate into the prototypical branchandbound algorithm. In the theoretical...
Nonlinear programming algorithms using trust regions and augmented Lagrangians with nonmonotone penalty parameters
, 1997
"... A model algorithm based on the successive quadratic programming method for solving the general nonlinear programming problem is presented. The objective function and the constraints of the problem are only required to be differentiable and their gradients to satisfy a Lipschitz condition. The strate ..."
Abstract

Cited by 23 (9 self)
 Add to MetaCart
A model algorithm based on the successive quadratic programming method for solving the general nonlinear programming problem is presented. The objective function and the constraints of the problem are only required to be differentiable and their gradients to satisfy a Lipschitz condition. The strategy for obtaining global convergence is based on the trust region approach. The merit function is a type of augmented Lagrangian. A new updating scheme is introduced for the penalty parameter, by means of which monotone increase is not necessary. Global convergence results are proved and numerical experiments are presented.
Curvilinear Stabilization Techniques for Truncated Newton Methods in Large Scale Unconstrained Optimization: the . . .
 SIAM J. Optim
, 1998
"... The aim of this paper is to define a new class of minimization algorithms for solving large scale unconstrained problems. In particular we describe a stabilization framework, based on a curvilinear linesearch, which uses a combination of a Newtontype direction and a negative curvature direction. Th ..."
Abstract

Cited by 23 (9 self)
 Add to MetaCart
(Show Context)
The aim of this paper is to define a new class of minimization algorithms for solving large scale unconstrained problems. In particular we describe a stabilization framework, based on a curvilinear linesearch, which uses a combination of a Newtontype direction and a negative curvature direction. The motivation for using negative curvature direction is that of taking into account local nonconvexity of the objective function. On the basis of this framework, we propose an algorithm which uses the Lanczos method for determining at each iteration both a Newtontype direction and an effective negative curvature direction. The results of an extensive numerical testing is reported together with a comparison with the LANCELOT package. These results show that the algorithm is very competitive and this seems to indicate that the proposed approach is promising. 1 Introduction In this work, we deal with the definition of new efficient unconstrained minimization algorithms for solving large scal...
Piecewise Sequential Quadratic Programming For Mathematical Programs With . . .
"... We describe some first and secondorder optimality conditions for mathematical programs with equilibrium constraints (MPEC). Mathematical programs with parametric nonlinear complementarity constraints are the focus. Of interest is the result that under a linear independence assumption that is stand ..."
Abstract

Cited by 18 (9 self)
 Add to MetaCart
We describe some first and secondorder optimality conditions for mathematical programs with equilibrium constraints (MPEC). Mathematical programs with parametric nonlinear complementarity constraints are the focus. Of interest is the result that under a linear independence assumption that is standard in nonlinear programming, the otherwise combinatorial problem of checking whether a point is stationary for an MPEC is reduced to checking stationarity of single nonlinear program. We also present a piecewise sequential quadratic programming (PSQP) algorithm for solving MPEC. Local quadratic convergence is shown under the linear independence assumption and a secondorder sufficient condition. Some computational results are given. KEY WORDS MPEC, bilevel program, nonlinear complementarity problem, nonlinear program, first and secondorder optimality conditions, linear independence constraint qualification, sequential quadratic programming, quadratic convergence. 2 Chapter 1 1 INTRODUC...
Nonmonotone curvilinear linesearch methods for unconstrained optimization
 Computational Optimization and Applications
, 1996
"... Abstract. We present a new algorithmic framework for solving unconstrained minimization problems that incorporates a curvilinear linesearch. The search direction used in our framework is a combination of an approximate Newton direction and a direction of negative curvature. Global convergence to a ..."
Abstract

Cited by 17 (6 self)
 Add to MetaCart
(Show Context)
Abstract. We present a new algorithmic framework for solving unconstrained minimization problems that incorporates a curvilinear linesearch. The search direction used in our framework is a combination of an approximate Newton direction and a direction of negative curvature. Global convergence to a stationary point where the Hessian matrix is positive semidefinite is exhibited for this class of algorithms by means of a nonmonotone stabilization strategy. An implementation using the BunchParlett decomposition is shown to outperform several other techniques on a large class of test problems. Keywords: 1.
LINEAR PROGRAMMING RELAXATIONS OF QUADRATICALLY CONSTRAINED QUADRATIC PROGRAMS
"... Abstract. We investigate the use of linear programming tools for solving semidefinite programming relaxations of quadratically constrained quadratic problems. Classes of valid linear inequalities are presented, including sparse PSD cuts, and principal minors PSD cuts. Computational results based on ..."
Abstract

Cited by 12 (1 self)
 Add to MetaCart
(Show Context)
Abstract. We investigate the use of linear programming tools for solving semidefinite programming relaxations of quadratically constrained quadratic problems. Classes of valid linear inequalities are presented, including sparse PSD cuts, and principal minors PSD cuts. Computational results based on instances from the literature are presented.
Convergence to Second Order Stationary Points in Inequality Constrained Optimization
 Mathematics of Operations Research
, 1998
"... : We propose a new algorithm for the nonlinear inequality constrained minimization problem, and prove that it generates a sequence converging to points satisfying the KKT second order necessary conditions for optimality. The algorithm is a line search algorithm using directions of negative curvature ..."
Abstract

Cited by 11 (5 self)
 Add to MetaCart
(Show Context)
: We propose a new algorithm for the nonlinear inequality constrained minimization problem, and prove that it generates a sequence converging to points satisfying the KKT second order necessary conditions for optimality. The algorithm is a line search algorithm using directions of negative curvature and it can be viewed as a non trivial extension of corresponding known techniques from unconstrained to constrained problems. The main tools employed in the definition and in the analysis of the algorithm are a differentiable exact penalty function and results from the theory of LC 1 functions. Key Words: Inequality constrained optimization, KKT second order necessary conditions, penalty function, LC 1 function, negative curvature direction. 1 Introduction We are concerned with the inequality constrained minimization problem (P) min f(x) s.t. g(x) 0; where f : IR n ! IR and g : IR n ! IR m are three times continuously differentiable. Our aim is to develope an algorithm that g...