Results 1  10
of
64
Benchmarking Optimization Software with Performance Profiles
, 2001
"... We propose performance profiles  distribution functions for a performance metric  as a tool for benchmarking and comparing optimization software. We show that performance profiles combine the best features of other tools for performance evaluation. 1 Introduction The benchmarking of optimi ..."
Abstract

Cited by 244 (7 self)
 Add to MetaCart
We propose performance profiles  distribution functions for a performance metric  as a tool for benchmarking and comparing optimization software. We show that performance profiles combine the best features of other tools for performance evaluation. 1 Introduction The benchmarking of optimization software has recently gained considerable visibility. Hans Mittlemann's [13] work on a variety of optimization software has frequently uncovered deficiencies in the software and has generally led to software improvements. Although Mittelmann's efforts have gained the most notice, other researchers have been concerned with the evaluation and performance of optimization codes. As recent examples, we cite [1, 2, 3, 4, 6, 12, 17]. The interpretation and analysis of the data generated by the benchmarking process are the main technical issues addressed in this paper. Most benchmarking efforts involve tables displaying the performance of each solver on each problem for a set of metrics such...
Trust region Newton method for largescale logistic regression
 In Proceedings of the 24th International Conference on Machine Learning (ICML
, 2007
"... Largescale logistic regression arises in many applications such as document classification and natural language processing. In this paper, we apply a trust region Newton method to maximize the loglikelihood of the logistic regression model. The proposed method uses only approximate Newton steps in ..."
Abstract

Cited by 64 (10 self)
 Add to MetaCart
Largescale logistic regression arises in many applications such as document classification and natural language processing. In this paper, we apply a trust region Newton method to maximize the loglikelihood of the logistic regression model. The proposed method uses only approximate Newton steps in the beginning, but achieves fast convergence in the end. Experiments show that it is faster than the commonly used quasi Newton approach for logistic regression. We also compare it with existing linear SVM implementations. 1
Automatic preconditioning by limited memory QuasiNewton updating
 SIAM J. Optim
"... The paper proposes a preconditioner for the conjugate gradient method (CG) that is designed for solving systems of equations Ax = bi with di erent right hand side vectors, or for solving a sequence of slowly varying systems Akx = bk. The preconditioner has the form of a limited memory quasiNewton m ..."
Abstract

Cited by 31 (2 self)
 Add to MetaCart
The paper proposes a preconditioner for the conjugate gradient method (CG) that is designed for solving systems of equations Ax = bi with di erent right hand side vectors, or for solving a sequence of slowly varying systems Akx = bk. The preconditioner has the form of a limited memory quasiNewton matrix and is generated using information from the CG iteration. The automatic preconditioner does not require explicit knowledge of the coe cient matrix A and is therefore suitable for problems where only products of A times avector can be computed. Numerical experiments indicate that the preconditioner has most to o er when these matrixvector products are expensive to compute, and when low accuracy in the solution is required. The e ectiveness of the preconditioner is tested within a Hessianfree Newton method for optimization, and by solving certain linear systems arising in nite element models.
LimitedMemory Matrix Methods with Applications
, 1997
"... Abstract. The focus of this dissertation is on matrix decompositions that use a limited amount of computer memory � thereby allowing problems with a very large number of variables to be solved. Speci�cally � we will focus on two applications areas � optimization and information retrieval. We introdu ..."
Abstract

Cited by 30 (6 self)
 Add to MetaCart
Abstract. The focus of this dissertation is on matrix decompositions that use a limited amount of computer memory � thereby allowing problems with a very large number of variables to be solved. Speci�cally � we will focus on two applications areas � optimization and information retrieval. We introduce a general algebraic form for the matrix update in limited�memory quasi� Newton methods. Many well�known methods such as limited�memory Broyden Family meth� ods satisfy the general form. We are able to prove several results about methods which sat� isfy the general form. In particular � we show that the only limited�memory Broyden Family method �using exact line searches � that is guaranteed to terminate within n iterations on an n�dimensional strictly convex quadratic is the limited�memory BFGS method. Further� more � we are able to introduce several new variations on the limited�memory BFGS method that retain the quadratic termination property. We also have a new result that shows that full�memory Broyden Family methods �using exact line searches � that skip p updates to the quasi�Newton matrix will terminate in no more than n�p steps on an n�dimensional strictly convex quadratic. We propose several new variations on the limited�memory BFGS method
Numerical methods for electronic structure calculations of materials
, 2006
"... The goal of this article is to give an overview of numerical problems encountered when determining the electronic structure of materials and the rich variety of techniques used to solve these problems. The paper is intended for a diverse scienti£c computing audience. For this reason, we assume the r ..."
Abstract

Cited by 15 (1 self)
 Add to MetaCart
The goal of this article is to give an overview of numerical problems encountered when determining the electronic structure of materials and the rich variety of techniques used to solve these problems. The paper is intended for a diverse scienti£c computing audience. For this reason, we assume the reader does not have an extensive background in the related physics. Our overview focuses on the nature of the numerical problems to be solved, their origin, and on the methods used to solve the resulting linear algebra or nonlinear optimization problems. It is common knowledge that the behavior of matter at the nanoscale is, in principle, entirely determined by the Schrödinger equation. In practice, this equation in its original form is not tractable. Successful, but approximate, versions of this equation, which allow one to study nontrivial systems, took about £ve or six decades to develop. In particular, the last two decades saw a ¤urry of activity in developing effective software. One of the main practical variants of the Schrödinger equation is based on what is referred to as Density Functional Theory (DFT). The combination of DFT with pseudopotentials allows one to obtain in an ef£cient way the ground state con£guration for many materials. This article will emphasize pseudopotentialdensity
Numerical experience with limitedMemory QuasiNewton methods and Truncated Newton methods
 SIAM J. Optimization
, 1992
"... Abstract. Computational experience with several limitedmemory quasiNewton and truncated Newton methods for unconstrained nonlinear optimization is described. Comparative tests were conducted on a wellknown test library [J. J. Mor, B. S. Garbow, and K. E. Hillstrom, ACM Trans. Math. Software, 7 (1 ..."
Abstract

Cited by 13 (9 self)
 Add to MetaCart
Abstract. Computational experience with several limitedmemory quasiNewton and truncated Newton methods for unconstrained nonlinear optimization is described. Comparative tests were conducted on a wellknown test library [J. J. Mor, B. S. Garbow, and K. E. Hillstrom, ACM Trans. Math. Software, 7 (1981), pp. 1741], on several synthetic problems allowing control of the clustering of eigenvalues in the Hessian spectrum, and on some largescale problems in oceanography and meteorology. The results indicate that among the tested limitedmemory quasiNewton methods, the LBFGS method [D. C. Liu and J. Nocedal, Math. Programming, 45 (1989), pp. 503528] has the best overall performance for the problems examined. The numerical performance of two truncated Newton methods, differing in the innerloop solution for the search vector, is competitive with that of LBFGS. Key words, limitedmemory quasiNewton methods, truncated Newton methods, synthetic cluster functions, largescale unconstrained minimization AMS subject classifications. 90C30, 93C20, 93C75, 65K10, 76C20 1. Introduction. Limitedmemory quasiNewton (LMQN) and truncated Newton
An overview of unconstrained optimization
 Online]. Available: citeseer.ist.psu.edu/fletcher93overview.html 150
, 1993
"... bundle filter method for nonsmooth nonlinear ..."
BFGS with update skipping and varying memory
 SIAM J. Optim
, 1998
"... Abstract. We give conditions under which limitedmemory quasiNewton methods with exact line searches will terminate in n steps when minimizing ndimensional quadratic functions. We show that although all Broyden family methods terminate in n steps in their fullmemory versions, only BFGS does so wi ..."
Abstract

Cited by 13 (3 self)
 Add to MetaCart
Abstract. We give conditions under which limitedmemory quasiNewton methods with exact line searches will terminate in n steps when minimizing ndimensional quadratic functions. We show that although all Broyden family methods terminate in n steps in their fullmemory versions, only BFGS does so with limitedmemory. Additionally, we show that fullmemory Broyden family methods with exact line searches terminate in at most n + p steps when p matrix updates are skipped. We introduce new limitedmemory BFGS variants and test them on nonquadratic minimization problems.
Inversion Based On Computational Simulations
, 1998
"... A standard approach to solving inversion problems that involve many parameters uses gradientbased optimization to find the parameters that best match the data. We will discuss enabling techniques that facilitate application of this approach to largescale computational simulations, which are the on ..."
Abstract

Cited by 12 (11 self)
 Add to MetaCart
A standard approach to solving inversion problems that involve many parameters uses gradientbased optimization to find the parameters that best match the data. We will discuss enabling techniques that facilitate application of this approach to largescale computational simulations, which are the only way to investigate many complex physical phenomena. Such simulations may not seem to lend themselves to calculation of the gradient with respect to numerous parameters. However, adjoint differentiation allows one to efficiently compute the gradient of an objective function with respect to all the variables of a simulation. When combined with advanced gradientbased optimization algorithms, adjoint differentiation permits one to solve very large problems of optimization or parameter estimation. These techniques will be illustrated through the simulation of the timedependent diffusion of infrared light through tissue, which has been used to perform optical tomography. The techniques discussed have a wide range of applicability to modeling including the optimization of models to achieve a desired design goal.
Application of a New Adjoint Newton Algorithm to the 3D ARPS Storm Scale Model Using Simulated Data
, 1997
"... The adjoint Newton algorithm (ANA) is based on the first and secondorder adjoint techniques allowing one to obtain the "Newton line search direction" by integrating a "tangent linear model" backward in time (with negative time steps). Moreover, the ANA provides a new technique to find "Newton line ..."
Abstract

Cited by 12 (1 self)
 Add to MetaCart
The adjoint Newton algorithm (ANA) is based on the first and secondorder adjoint techniques allowing one to obtain the "Newton line search direction" by integrating a "tangent linear model" backward in time (with negative time steps). Moreover, the ANA provides a new technique to find "Newton line search direction" without using gradient information. The error present in approximating the Hessian (the matrix of second order derivatives) of the cost function with respect to the control variables in the quasiNewton type algorithm is thus completely eliminated, while the storage problem related to storing the Hessian no longer exists since the explicit Hessian is not required in this algorithm. The ANA is applied here, for the first time, in the framework of 4D variational data assimilation to the adiabatic version of the Advanced Regional Prediction System (ARPS), a 3dimensional, compressible, nonhydrostatic stormscale model. The purpose is to assess the feasibility and efficiency ...