Results 1  10
of
14
On the limited memory BFGS method for large scale optimization
 Mathematical Programming
, 1989
"... this paper has appeared in ..."
Theory and implementation of numerical methods based on RungeKutta integration for solving optimal control problems
, 1996
"... ..."
Hybrid deterministicstochastic methods for data fitting
, 2011
"... Abstract. Many structured datafitting applications require the solution of an optimization problem involving a sum over a potentially large number of measurements. Incremental gradient algorithms offer inexpensive iterations by sampling a subset of the terms in the sum. These methods can make great ..."
Abstract

Cited by 9 (4 self)
 Add to MetaCart
Abstract. Many structured datafitting applications require the solution of an optimization problem involving a sum over a potentially large number of measurements. Incremental gradient algorithms offer inexpensive iterations by sampling a subset of the terms in the sum. These methods can make great progress initially, but often slow as they approach a solution. In contrast, fullgradient methods achieve steady convergence at the expense of evaluating the full objective and gradient on each iteration. We explore hybrid methods that exhibit the benefits of both approaches. Rateofconvergence analysis shows that by controlling the sample size in an incremental gradient algorithm, it is possible to maintain the steady convergence rates of fullgradient methods. We detail a practical quasiNewton implementation based on this approach. Numerical experiments illustrate its potential benefits. 1. Introduction. Data
Measures For Symmetric RankOne Updates
 Department of Combinatorics and Optimization, University of Waterloo
, 1990
"... Measures of deviation of a symmetric positive definite matrix from the identity are derived. They give rise to symmetric rankone, SR1, type updates. The measures are motivated by considering the volume of the symmetric difference of the two ellipsoids, which arise from the current and updated quadr ..."
Abstract

Cited by 6 (2 self)
 Add to MetaCart
Measures of deviation of a symmetric positive definite matrix from the identity are derived. They give rise to symmetric rankone, SR1, type updates. The measures are motivated by considering the volume of the symmetric difference of the two ellipsoids, which arise from the current and updated quadratic models in quasiNewton methods. The measure defined by the problem  maximize the determinant subject to a bound of 1 on the largest eigenvalue  yields the SR1 update. The measure oe(A) = 1 (A) det(A) 1 n yields the optimally conditioned, sized, symmetric rankone updates, [1, 2]. The volume considerations also suggest a `correction' for the initial stepsize for these sized updates. It is then shown that the oe optimal updates, as well as the OrenLuenberger selfscaling updates [3], are all optimal updates for the measure, the ` 2 condition number. Moreover, all four sized updates result in the same largest (and smallest) 'scaled' eigenvalue and corresponding eigenvector. In fac...
A Merit Function for Inequality Constrained Nonlinear Programming Problems
 Internal Report 4702, National Institute of Standards and Technology
, 1993
"... We consider the use of the sequential quadratic programming (SQP) technique for solving the inequality constrained minimization problem min x f(x) subject to: g i (x) 0; i = 1; : : : ; m: SQP methods require the use of an auxiliary function, called a merit function or linesearch function, for asse ..."
Abstract

Cited by 5 (4 self)
 Add to MetaCart
We consider the use of the sequential quadratic programming (SQP) technique for solving the inequality constrained minimization problem min x f(x) subject to: g i (x) 0; i = 1; : : : ; m: SQP methods require the use of an auxiliary function, called a merit function or linesearch function, for assessing the steps that are generated. We derive a merit function by adding slack variables to create an equality constrained problem and then using the merit function developed earlier by the authors for the equality constrained case. We stress that we do not solve the slack variable problem, but only use it to construct the merit function. The resulting function is simplified in a certain way that leads to an effective procedure for updating the squares of the slack variables. A globally convergent algorithm, based on this merit function, is suggested, and is demonstrated to be effective in practice. Contribution of the National Institute of Standards and Technology and not subject to copyr...
An Alternating MultiStep QuasiNewton Method for Unconstrained Optimization
 J. Comput. Appl. Math
, 1997
"... We consider multistep quasiNewton methods for unconstrained optimization. These methods were introduced by the authors [6, 7, 8], who showed how an interpolating curve in the variablespace could be used to derive an appropriate generalization of the Secant Equation normally employed in the constr ..."
Abstract

Cited by 4 (2 self)
 Add to MetaCart
We consider multistep quasiNewton methods for unconstrained optimization. These methods were introduced by the authors [6, 7, 8], who showed how an interpolating curve in the variablespace could be used to derive an appropriate generalization of the Secant Equation normally employed in the construction of quasiNewton methods. One of the most successful of these multistep methods employs the current approximation to the Hessian to determine the parametrization of the interpolating curve and, hence, the derivatives which are required in the generalized updating formula. However, certain approximations were found to be necessary in the process, in order to reduce the level of computation required (which must be repeated at each iteration) to acceptable levels. In this paper, we show how a variant of this algorithm, which avoids the need for such approximations, may be obtained. This is accomplished by alternating, on successive iterations, a singlestep and a twostep method. The res...
New Implicit Updates in Multistep QuasiNewton Methods for Unconstrained Optimisation
 Comput. Math. Appl
, 2001
"... Multistep quasiNewton methods for optimisation (using data from more than one previous step to revise the current approximate Hessian) were introduced by Ford and Moghrabi in [4], where they showed how to construct such methods by means of interpolating curves. These methods also utilise standard ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
Multistep quasiNewton methods for optimisation (using data from more than one previous step to revise the current approximate Hessian) were introduced by Ford and Moghrabi in [4], where they showed how to construct such methods by means of interpolating curves. These methods also utilise standard quasiNewton formulae, but with the vectors normally employed in the formulae replaced by others determined from a multistep version of the Secant Equation. Some methods (the `Accumulative' and `FixedPoint' approaches) for de ning the parameter values which correspond to the iterates on the interpolating curve were presented in [5]. Both the Accumulative and the FixedPoint methods measure the distances required to parameterise the interpolating polynomials via a norm de ned by a positivede nite matrix M . The FixedPoint algorithm which takes M to be the current approximate Hessian was found, experimentally, to be the best of the six multistep methods studied in [5] (all of which exhibited improved numerical performance by comparison with the standard singlestep BFGS method).
Measures For Least Change Secant Methods
, 1992
"... In order to solve an unconstrained optimization problem, quasiNewton methods are used when the Hessian matrix of the objective function can not be cheaply computed. In these methods, at each step, the current approximation to the Hessian matrix is updated to a new approximation satisfying the secan ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
In order to solve an unconstrained optimization problem, quasiNewton methods are used when the Hessian matrix of the objective function can not be cheaply computed. In these methods, at each step, the current approximation to the Hessian matrix is updated to a new approximation satisfying the secant equation. The open question is how can we find a new approximation which maintains as much information as possible already built up in the current approximation. In this thesis we try to answer this question by considering various measures related to the eigenvalues of the scaled new approximation. The analysis of these eigenvalues plays an important role in our approach. The measures can be obtained by considering the matrix condition numbers, the volume of the symmetric difference of the ellipsoids corresponding to the Hessian approximation and other concepts. We show that all the ! , oe , and ΓΈ optimal updates are also the optimally conditioned updates. Multicriteria provides means to ...
A Nonlinear Model for Functionvalue Multistep Methods
, 1998
"... We develop a framework (employing scaling functions) for the construction of multistep quasiNewton methods (for unconstrained optimization) which utilize values of the objective function. The methods are constructed via interpolants of the m+ 1 most recent iterates / gradient evaluations, and poss ..."
Abstract
 Add to MetaCart
We develop a framework (employing scaling functions) for the construction of multistep quasiNewton methods (for unconstrained optimization) which utilize values of the objective function. The methods are constructed via interpolants of the m+ 1 most recent iterates / gradient evaluations, and possess a free parameter which introduces an additional degree of flexibility. This permits the interpolating polynomials to assimilate information (in the form of functionvalues) which is readily available at each iteration. This information is incorporated in updating the Hessian approximation at each iteration, in an attempt to accelerate convergence. We concentrate on a specific example from the general family of methods, corresponding to a particular choice of the scaling function, and from it derive three new algorithms. The relative numerical performance of these methods is assessed, and the most successful of them is then compared with the standard BFGS method and with an earlier algori...