Results 1  10
of
10
On the limited memory BFGS method for large scale optimization
 Mathematical Programming
, 1989
"... this paper has appeared in ..."
LimitedMemory Matrix Methods with Applications
, 1997
"... Abstract. The focus of this dissertation is on matrix decompositions that use a limited amount of computer memory � thereby allowing problems with a very large number of variables to be solved. Speci�cally � we will focus on two applications areas � optimization and information retrieval. We introdu ..."
Abstract

Cited by 30 (6 self)
 Add to MetaCart
Abstract. The focus of this dissertation is on matrix decompositions that use a limited amount of computer memory � thereby allowing problems with a very large number of variables to be solved. Speci�cally � we will focus on two applications areas � optimization and information retrieval. We introduce a general algebraic form for the matrix update in limited�memory quasi� Newton methods. Many well�known methods such as limited�memory Broyden Family meth� ods satisfy the general form. We are able to prove several results about methods which sat� isfy the general form. In particular � we show that the only limited�memory Broyden Family method �using exact line searches � that is guaranteed to terminate within n iterations on an n�dimensional strictly convex quadratic is the limited�memory BFGS method. Further� more � we are able to introduce several new variations on the limited�memory BFGS method that retain the quadratic termination property. We also have a new result that shows that full�memory Broyden Family methods �using exact line searches � that skip p updates to the quasi�Newton matrix will terminate in no more than n�p steps on an n�dimensional strictly convex quadratic. We propose several new variations on the limited�memory BFGS method
BFGS with update skipping and varying memory
 SIAM J. Optim
, 1998
"... Abstract. We give conditions under which limitedmemory quasiNewton methods with exact line searches will terminate in n steps when minimizing ndimensional quadratic functions. We show that although all Broyden family methods terminate in n steps in their fullmemory versions, only BFGS does so wi ..."
Abstract

Cited by 11 (2 self)
 Add to MetaCart
Abstract. We give conditions under which limitedmemory quasiNewton methods with exact line searches will terminate in n steps when minimizing ndimensional quadratic functions. We show that although all Broyden family methods terminate in n steps in their fullmemory versions, only BFGS does so with limitedmemory. Additionally, we show that fullmemory Broyden family methods with exact line searches terminate in at most n + p steps when p matrix updates are skipped. We introduce new limitedmemory BFGS variants and test them on nonquadratic minimization problems.
On Trust Region Methods for Unconstrained Minimization Without Derivatives
, 2002
"... We consider some algorithms for unconstrained minimization without derivatives that form linear or quadratic models by interpolation to values of the objective function. Then a new vector of variables is calculated by minimizing the current model within a trust region. Techniques are described for a ..."
Abstract

Cited by 9 (1 self)
 Add to MetaCart
We consider some algorithms for unconstrained minimization without derivatives that form linear or quadratic models by interpolation to values of the objective function. Then a new vector of variables is calculated by minimizing the current model within a trust region. Techniques are described for adjusting the trust region radius, and for choosing positions of the interpolation points that maintain not only nonsingularity of the interpolation equations but also the adequacy of the model. Particular attention is given to quadratic models with diagonal second derivative matrices, because numerical experiments show that they are often more efficient than full quadratic models for general objective functions. Finally, some recent research on the updating of full quadratic models is described briefly, using fewer interpolation equations than before. The resultant freedom is taken up by minimizing the Frobenius norm of the change to the second derivative matrix of the model. A preliminary version of this method provides some very promising numerical results.
Parallel Algorithms for Largescale Nonlinear Optimization
 the Journal of International Transactions in Operational Research
, 1998
"... Multistep, multidirectional parallel variable metric... ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
Multistep, multidirectional parallel variable metric...
Sparsity in Higher Order Methods in Optimization
, 2006
"... In this paper it is shown that when the sparsity structure of the problem is utilized higher order methods are competitive to second order methods (Newton), for solving unconstrained optimization problems when the objective function is three times continuously differentiable. It is also shown how to ..."
Abstract

Cited by 1 (1 self)
 Add to MetaCart
In this paper it is shown that when the sparsity structure of the problem is utilized higher order methods are competitive to second order methods (Newton), for solving unconstrained optimization problems when the objective function is three times continuously differentiable. It is also shown how to arrange the computations of the higher derivatives. 1
A Memetic Algorithm Assisted by an Adaptive Topology RBF Network and Variable Local Models for Expensive Optimization Problems
"... A common practice in modern engineering is that of simulationdriven optimization. This implies replacing costly and lengthy laboratory experiments with computer experiments, i.e. computationallyintensive simulations which model real world physics with high fidelity. Due to the complexity of such s ..."
Abstract
 Add to MetaCart
A common practice in modern engineering is that of simulationdriven optimization. This implies replacing costly and lengthy laboratory experiments with computer experiments, i.e. computationallyintensive simulations which model real world physics with high fidelity. Due to the complexity of such simulations a single simulation run can require up to
SelfScaling Parallel QuasiNewton Methods
"... In this paper, a new class of selfscaling quasiNewton#SSQN# updates for solving unconstrained nonlinear optimization problems#UNOPs# is proposed. It is shown that many existing QN updates can be considered as special cases of the new family.Parallel SSQN algorithms based on this class of class of ..."
Abstract
 Add to MetaCart
In this paper, a new class of selfscaling quasiNewton#SSQN# updates for solving unconstrained nonlinear optimization problems#UNOPs# is proposed. It is shown that many existing QN updates can be considered as special cases of the new family.Parallel SSQN algorithms based on this class of class of updates are studied. In comparison to standard serial QN methods, proposed parallel SSQN#SSPQN# algorithms show signi#cant improvement in the total number of iterations and function#gradientevaluations required in solving a wide range of test problems. In fact, the average speedup factors by the new SSPQN algorithms over the conventional BFGS method and E04DGE in NAG library are 3:22=3:13 and 2:80=3:09 , respectively#in terms of total number of iterations and total number of function #gradientevaluations required#. For some test problems, the speedup factor gained by the new algorithms can be as high as 25#25 over BFGS and 20#25 over E04DGE , both in terms of total number of iterations and function#gradientevaluations. 1
NorthHolland QNLIKE VARIABLE STORAGE CONJUGATE GRADIENTS*
, 1982
"... Both conjugate gradient and quasiNewton methods are quite successful at minimizing smooth nonlinear functions of several variables, and each has its advantages. In particular, conjugate gradient methods require much less storage to implement than a quasiNewton code and therefore find application w ..."
Abstract
 Add to MetaCart
Both conjugate gradient and quasiNewton methods are quite successful at minimizing smooth nonlinear functions of several variables, and each has its advantages. In particular, conjugate gradient methods require much less storage to implement than a quasiNewton code and therefore find application when storage limitations occur. They are, however, slower, so there have recently been attempts to combine CG and QN algorithms o as to obtain an algorithm with good convergence properties and low storage requirements. One such method is the code CONMIN clue to Shanno and Phua; it has proven quite successful but it has one limitation. It has no middle ground, in that it either operates as a quasiNewton code using O(n ~) storage locations, or as a conjugate gradient code using 7n locations, but it cannot take advantage of the not unusual situation where more than 7n locations are available, but a quasiNewton code requires an excessive amount of storage. In this paper we present a way of looking at conjugate gradient algorithms which was in fact given by Shanno and Phua but which we carry further, emphasize and clarify. This applies in particular to Beale's 3term recurrence relation. Using this point of view, we develop a new combined CGQN algorithm which can use whatever storage is available; CONMIN occurs as a special case. We present numerical results to demonstrate that the new algorithm is never worse than CONMIN and that it is almost always better if even a small amount of extra storage is provided.