this paper has appeared in

We derive compact representations of BFGS and symmetric rank-one matrices for optimization. These representations allow us to efficiently implement limited memory methods for large constrained optimization problems. In particular, we discuss how to compute projections of limited memory matrices onto subspaces. We also present a compact representation of the matrices generated by Broyden's update for solving systems of nonlinear equations. Key words: Quasi-Newton method, constrained optimization, limited memory method, large-scale optimization. Abbreviated title: Representation of quasi-Newton matrices. 1. Introduction. Limited memory quasi-Newton methods are known to be effective techniques for solving certain classes of large-scale unconstrained optimization problems (Buckley and Le Nir (1983), Liu and Nocedal (1989), Gilbert and Lemar'echal (1989)) . They make simple approximations of Hessian matrices, which are often good enough to provide a fast rate of linear convergence, and re...

this article I will attempt to review the most recent advances in the theory of unconstrained optimization, and will also describe some important open questions. Before doing so, I should point out that the value of the theory of optimization is not limited to its capacity for explaining the behavior of the most widely used techniques. The question

L-BFGS-B is a limited memory algorithm for solving large nonlinear optimization problems subject to simple bounds on the variables. It is intended for problems in which information on the Hessian matrix is difficult to obtain, or for large dense problems. L-BFGS-B can also be used for unconstrained problems, and in this case performs similarly to its predecessor, algorithm L-BFGS (Harwell routine VA15). The algorithm is implemented in Fortran 77. Categories and Subject Descriptors: G.1.6 [Numerical Analysis]: Optimization -- gradient methods; G.4 [Mathematics of Computing]: Mathematical Software. General Terms: Algorithms Additional Key Words and Phrases: variable metric method, large scale optimization, nonlinear optimization, limited memory method. 1 Department of Electrical Engineering and Computer Science, Northwestern University, Evanston Il 60208. These authors were supported by National Science Foundation Grants CCR-9101359 and ASC-9213149, and by Department of Energy Grant DE-...

The paper proposes a preconditioner for the conjugate gradient method (CG) that is designed for solving systems of equations Ax = bi with di erent right hand side vectors, or for solving a sequence of slowly varying systems Akx = bk. The preconditioner has the form of a limited memory quasi-Newton matrix and is generated using information from the CG iteration. The automatic preconditioner does not require explicit knowledge of the coe cient matrix A and is therefore suitable for problems where only products of A times avector can be computed. Numerical experiments indicate that the preconditioner has most to o er when these matrix-vector products are expensive to compute, and when low accuracy in the solution is required. The e ectiveness of the preconditioner is tested within a Hessian-free Newton method for optimization, and by solving certain linear systems arising in nite element models.

Optimal control theory is applied to a variational data assimilation problem in the context of the assimilation of altimeter data in a quasigeostrophic ocean model. Related to the issue of the minimization of the cost function, a sensitivity analysis is applied to the optimality system to derive the sensitivity of the retrieved control variable ( here the initial condition) with respect to the observations. The derivation of the sensitivity of a response function in the case of data assimilation is reviewed and a new method of performing the derivation of this sensitivity is proposed. 1 1

A set of four-dimensional variational data assimilation (4D-Var) experiments were conducted using both a standard method and an incremental method in an identical twin framework. The full physics adjoint model of the Florida State University global spectral model (FSUGSM) was used in the standard 4D-Var, while the adjoint of only a few selected physical parameterizations was used in the incremental method. The impact of physical processes on 4D-Var was examined in detail by comparing the results of these experiments. The inclusion of full physics turned out to be significantly beneficial in terms of assimilation error to the lower troposphere during the entire minimization process. The beneficial impact was found to be primarily related to boundary layer physics. The precipitation physics in the adjoint model also tended to have a beneficial impact after an intermediate number (50) of minimization iterations. Experiment results confirmed that the forecast from assimilation analyses with the full physics adjoint model displays a shorter precipitation spinup period. The beneficial impact on precipitation spinup did not result solely from the inclusion of the precipitation physics in the adjoint model, but rather from the combined impact of several physical processes. The inclusion of full physics in the adjoint model exhibited a detrimental impact on the rate of convergence at an early stage of the minimization process, but did not affect the final convergence.

A new method for solving large nonlinear optimization problems is outlined. It attempts to combine the best properties of the discrete-truncated Newton method and the limited memory BFGS method, to produce an algorithm that is both economical and capable of handling ill-conditioned problems. The key idea is to use the curvature information generated during the computation of the discrete Newton step to improvethelimited memory BFGS approximations. The numerical performance of the new method is studied using a family of functions whose nonlinearity and condition number can be controlled.

This paper reviews advances in Newton, quasi-Newton and conjugate gradient methods for large scale optimization. It also describes several packages developed during the last ten years, and illustrates their performance on some practical problems. Much attention is given to the concept of partial separabilitywhich is gaining importance with the arrival of automatic differentiation tools and of optimization software that fully exploits its properties.

Abstract not found