this paper has appeared in

Abstract not found

We consider the use of the sequential quadratic programming (SQP) technique for solving the inequality constrained minimization problem min x f(x) subject to: g i (x) 0; i = 1; : : : ; m: SQP methods require the use of an auxiliary function, called a merit function or line-search function, for assessing the steps that are generated. We derive a merit function by adding slack variables to create an equality constrained problem and then using the merit function developed earlier by the authors for the equality constrained case. We stress that we do not solve the slack variable problem, but only use it to construct the merit function. The resulting function is simplified in a certain way that leads to an effective procedure for updating the squares of the slack variables. A globally convergent algorithm, based on this merit function, is suggested, and is demonstrated to be effective in practice. Contribution of the National Institute of Standards and Technology and not subject to copyr...

Abstract. Many structured data-fitting applications require the solution of an optimization problem involving a sum over a potentially large number of measurements. Incremental gradient algorithms offer inexpensive iterations by sampling a subset of the terms in the sum. These methods can make great progress initially, but often slow as they approach a solution. In contrast, full-gradient methods achieve steady convergence at the expense of evaluating the full objective and gradient on each iteration. We explore hybrid methods that exhibit the benefits of both approaches. Rate-ofconvergence analysis shows that by controlling the sample size in an incremental gradient algorithm, it is possible to maintain the steady convergence rates of full-gradient methods. We detail a practical quasi-Newton implementation based on this approach. Numerical experiments illustrate its potential benefits. 1. Introduction. Data

Measures of deviation of a symmetric positive definite matrix from the identity are derived. They give rise to symmetric rank-one, SR1, type updates. The measures are motivated by considering the volume of the symmetric difference of the two ellipsoids, which arise from the current and updated quadratic models in quasi-Newton methods. The measure defined by the problem - maximize the determinant subject to a bound of 1 on the largest eigenvalue - yields the SR1 update. The measure oe(A) = 1 (A) det(A) 1 n yields the optimally conditioned, sized, symmetric rank-one updates, [1, 2]. The volume considerations also suggest a `correction' for the initial stepsize for these sized updates. It is then shown that the oe -optimal updates, as well as the Oren-Luenberger self-scaling updates [3], are all optimal updates for the measure, the ` 2 condition number. Moreover, all four sized updates result in the same largest (and smallest) 'scaled' eigenvalue and corresponding eigenvector. In fac...

We consider multi-step quasi-Newton methods for unconstrained optimization. These methods were introduced by the authors [6, 7, 8], who showed how an interpolating curve in the variable-space could be used to derive an appropriate generalization of the Secant Equation normally employed in the construction of quasi-Newton methods. One of the most successful of these multi-step methods employs the current approximation to the Hessian to determine the parametrization of the interpolating curve and, hence, the derivatives which are required in the generalized updating formula. However, certain approximations were found to be necessary in the process, in order to reduce the level of computation required (which must be repeated at each iteration) to acceptable levels. In this paper, we show how a variant of this algorithm, which avoids the need for such approximations, may be obtained. This is accomplished by alternating, on successive iterations, a single-step and a two-step method. The res...

Multi-step quasi-Newton methods for optimisation (using data from more than one previous step to revise the current approximate Hessian) were introduced by Ford and Moghrabi in [4], where they showed how to construct such methods by means of interpolating curves. These methods also utilise standard quasi-Newton formulae, but with the vectors normally employed in the formulae replaced by others determined from a multi-step version of the Secant Equation. Some methods (the `Accumulative' and `Fixed-Point' approaches) for de ning the parameter values which correspond to the iterates on the interpolating curve were presented in [5]. Both the Accumulative and the Fixed-Point methods measure the distances required to parameterise the interpolating polynomials via a norm de ned by a positive-de nite matrix M . The Fixed-Point algorithm which takes M to be the current approximate Hessian was found, experimentally, to be the best of the six multi-step methods studied in [5] (all of which exhibited improved numerical performance by comparison with the standard single-step BFGS method).

In order to solve an unconstrained optimization problem, quasi-Newton methods are used when the Hessian matrix of the objective function can not be cheaply computed. In these methods, at each step, the current approximation to the Hessian matrix is updated to a new approximation satisfying the secant equation. The open question is how can we find a new approximation which maintains as much information as possible already built up in the current approximation. In this thesis we try to answer this question by considering various measures related to the eigenvalues of the scaled new approximation. The analysis of these eigenvalues plays an important role in our approach. The measures can be obtained by considering the matrix condition numbers, the volume of the symmetric difference of the ellipsoids corresponding to the Hessian approximation and other concepts. We show that all the ! , oe , and ø optimal updates are also the optimally conditioned updates. Multi-criteria provides means to ...

We develop a framework (employing scaling functions) for the construction of multi-step quasi-Newton methods (for unconstrained optimization) which utilize values of the objective function. The methods are constructed via interpolants of the m+ 1 most recent iterates / gradient evaluations, and possess a free parameter which introduces an additional degree of flexibility. This permits the interpolating polynomials to assimilate information (in the form of function-values) which is readily available at each iteration. This information is incorporated in updating the Hessian approximation at each iteration, in an attempt to accelerate convergence. We concentrate on a specific example from the general family of methods, corresponding to a particular choice of the scaling function, and from it derive three new algorithms. The relative numerical performance of these methods is assessed, and the most successful of them is then compared with the standard BFGS method and with an earlier algori...

Abstract not found