Results 11  20
of
153
A BoxConstrained Optimization Algorithm With Negative Curvature Directions and Spectral Projected Gradients
, 2001
"... A practical algorithm for boxconstrained optimization is introduced. The algorithm combines an activeset strategy with spectral projected gradient iterations. In the interior of each face a strategy that deals eciently with negative curvature is employed. Global convergence results are given. ..."
Abstract

Cited by 26 (5 self)
 Add to MetaCart
A practical algorithm for boxconstrained optimization is introduced. The algorithm combines an activeset strategy with spectral projected gradient iterations. In the interior of each face a strategy that deals eciently with negative curvature is employed. Global convergence results are given. Numerical results are presented. Keywords: box constrained minimization, active set methods, spectral projected gradients, dogleg path methods. AMS Subject Classication: 49M07, 49M10, 65K, 90C06, 90C20. 1
A new active set algorithm for box constrained optimization
 SIAM Journal on Optimization
"... ..."
(Show Context)
Optimizing costly functions with simple constraints: A limitedmemory projected quasinewton algorithm
 Proc. of Conf. on Artificial Intelligence and Statistics
, 2009
"... An optimization algorithm for minimizing a smooth function over a convex set is described. Each iteration of the method computes a descent direction by minimizing, over the original constraints, a diagonal plus lowrank quadratic approximation to the function. The quadratic approximation is construct ..."
Abstract

Cited by 25 (7 self)
 Add to MetaCart
An optimization algorithm for minimizing a smooth function over a convex set is described. Each iteration of the method computes a descent direction by minimizing, over the original constraints, a diagonal plus lowrank quadratic approximation to the function. The quadratic approximation is constructed using a limitedmemory quasiNewton update. The method is suitable for largescale problems where evaluation of the function is substantially more expensive than projection onto the constraint set. Numerical experiments on onenorm regularized test problems indicate that the proposed method is competitive with stateoftheart methods such as boundconstrained LBFGS and orthantwise descent. We further show that the method generalizes to a wide class of problems, and substantially improves on stateoftheart methods for problems such as learning the structure of Gaussian graphical models and Markov random fields. 1
On the BarzilaiBorwein method
, 2001
"... A review is given of the underlying theory and recent developments in regard to the BarzilaiBorwein steepest descent method for large scale unconstrained optimization. One aim is to assess why the method seems to be comparable in practical eciency to conjugate gradient methods. The importance of ..."
Abstract

Cited by 24 (1 self)
 Add to MetaCart
A review is given of the underlying theory and recent developments in regard to the BarzilaiBorwein steepest descent method for large scale unconstrained optimization. One aim is to assess why the method seems to be comparable in practical eciency to conjugate gradient methods. The importance of using a nonmonotone line search is stressed, although some suggestions are made as to why the modi cation proposed by Raydan [22] often does not usually perform well for an illconditioned problem. Extensions for box constraints are discussed. A number of interesting open questions are put forward. Keywords BarzilaiBorwein method, steepest descent, elliptic systems, unconstrained optimization. 1
Gradient projection methods for quadratic programs and applications in training support vector machines
 Optim. Methods Softw
, 2005
"... Gradient projection methods based on the BarzilaiBorwein spectral steplength choices are considered for quadratic programming problems with simple constraints. Wellknown nonmonotone spectral projected gradient methods and variable projection methods are discussed. For both approaches the behavior ..."
Abstract

Cited by 23 (6 self)
 Add to MetaCart
(Show Context)
Gradient projection methods based on the BarzilaiBorwein spectral steplength choices are considered for quadratic programming problems with simple constraints. Wellknown nonmonotone spectral projected gradient methods and variable projection methods are discussed. For both approaches the behavior of different combinations of the two spectral steplengths is investigated. A new adaptive steplength alternating rule is proposed that becomes the basis for a generalized version of the variable projection method (GVPM). Convergence results are given for the proposed approach and its effectiveness is shown by means of an extensive computational study on several test problems, including the special quadratic programs arising in training support vector machines. Finally, the GVPM behavior as inner QP solver in decomposition techniques for largescale support vector machines is also evaluated.
Restricted optimization: a clue to a fast and accurate implementation of the Common Reflection Surface Stack method
 JOURNAL OF APPLIED GEOPHYSICS
, 1999
"... For a fixed, central ray in an isotropic elastic or acoustic media, traveltime moveouts of rays in its vicinity can be described in terms of a certain number of parameters that refer to the central ray only. The determination of these parameters out of multicoverage data leads to very powerful al ..."
Abstract

Cited by 23 (10 self)
 Add to MetaCart
For a fixed, central ray in an isotropic elastic or acoustic media, traveltime moveouts of rays in its vicinity can be described in terms of a certain number of parameters that refer to the central ray only. The determination of these parameters out of multicoverage data leads to very powerful algorithms that can be used for several imaging and inversion processes. Assuming twodimensional propagation, the traveltime expressions depend on three parameters directly related to the geometry of the unknown model in the vicinity of the central ray. We present a new method to extract these parameters out of coherency analysis applied directly to the data. It uses (a) fast oneparameter searches on different sections extracted from the multicoverage data to derive initial values of the sections parameters, and (b) the application of a recently introduced Spectral Projected Gradient optimization algorithm for the final parameter estimation. Application of the method on a synthetic example shows an excellent performance of the algorithm both in accuracy and efficiency. The results obtained so far indicate that the algorithm may be a feasible option to solve the corresponding, harder, full threedimensional problem, in which eight parameters, instead of three, are required.
Structure Learning of Bayesian Networks using Constraints
"... This paper addresses exact learning of Bayesian network structure from data and expert’s knowledge based on score functions that are decomposable. First, it describes useful properties that strongly reduce the time and memory costs of many known methods such as hillclimbing, dynamic programming and ..."
Abstract

Cited by 19 (1 self)
 Add to MetaCart
(Show Context)
This paper addresses exact learning of Bayesian network structure from data and expert’s knowledge based on score functions that are decomposable. First, it describes useful properties that strongly reduce the time and memory costs of many known methods such as hillclimbing, dynamic programming and sampling variable orderings. Secondly, a branch and bound algorithm is presented that integrates parameter and structural constraints with data in a way to guarantee global optimality with respect to the score function. It is an anytime procedure because, if stopped, it provides the best current solution and an estimation about how far it is from the global solution. We show empirically the advantages of the properties and the constraints, and the applicability of the algorithm to large data sets (up to one hundred variables) that cannot be handled by other current methods (limited to around 30 variables). 1.
Spectral residual method without gradient information for solving largescale nonlinear systems: Theory and experiments
, 2004
"... Abstract. A fully derivativefree spectral residual method for solving largescale nonlinear systems of equations is presented. It uses in a systematic way the residual vector as a search direction, a spectral steplength that produces a nonmonotone process and a globalization strategy that allows for ..."
Abstract

Cited by 18 (5 self)
 Add to MetaCart
(Show Context)
Abstract. A fully derivativefree spectral residual method for solving largescale nonlinear systems of equations is presented. It uses in a systematic way the residual vector as a search direction, a spectral steplength that produces a nonmonotone process and a globalization strategy that allows for this nonmonotone behavior. The global convergence analysis of the combined scheme is presented. An extensive set of numerical experiments that indicate that the new combination is competitive and frequently better than wellknown NewtonKrylov methods for largescale problems is also presented. 1.
On the Asymptotic Behaviour of some New Gradient Methods
 Mathematical Programming
, 2003
"... The BarzilaiBorwein (BB) gradient method, and some other new gradient methods have shown themselves to be competitive with conjugate gradient methods for solving large dimension nonlinear unconstrained optimization problems. Little is known about the asymptotic behaviour, even when applied to n ..."
Abstract

Cited by 18 (4 self)
 Add to MetaCart
(Show Context)
The BarzilaiBorwein (BB) gradient method, and some other new gradient methods have shown themselves to be competitive with conjugate gradient methods for solving large dimension nonlinear unconstrained optimization problems. Little is known about the asymptotic behaviour, even when applied to n dimensional quadratic functions, except in the case that n = 2. We show in the quadratic case how it is possible to compute this asymptotic behaviour, and observe that as n increases there is a transition from superlinear to linear convergence at some value of n 4, depending on the method. By neglecting certain terms in the recurrence relations we de ne simpli ed versions of the methods, which are able to predict this transition. The simpli ed methods also predict that for larger values of n, the eigencomponents of the gradient vectors converge in modulus to a common value, which is a similar to a property observed to hold in the real methods. Some unusual and interesting recurrence relations are analysed in the course of the study.
A Spectral Conjugate Gradient Method for Unconstrained Optimization
, 1999
"... A family of scaled conjugategradient algorithms for largescale unconstrained minimization is dened. The Perry, the PolakRibiere and the FletcherReeves formulae are compared using a spectral scaling derived from Raydan's spectral gradient optimization method. The best combination of form ..."
Abstract

Cited by 18 (2 self)
 Add to MetaCart
A family of scaled conjugategradient algorithms for largescale unconstrained minimization is dened. The Perry, the PolakRibiere and the FletcherReeves formulae are compared using a spectral scaling derived from Raydan's spectral gradient optimization method. The best combination of formula, scaling and initial choice of steplength is compared against well known algorithms using a classical set of problems. An additional comparison involving an illconditioned estimation problem in Optics is presented. Keywords. Unconstrained minimization, spectral gradient method, conjugate gradients. AMS: 49M07, 49M10, 90C06, 65K. 1