Abstract

this article I will attempt to review the most recent advances in the theory of unconstrained optimization, and will also describe some important open questions. Before doing so, I should point out that the value of the theory of optimization is not limited to its capacity for explaining the behavior of the most widely used techniques. The question

Citations

1128	Practical Optimization - Gill, Murray, et al. - 1981
920	The Algebraic Eigenvalue Problem - Wilkinson - 1965
312	On the limited memory BFGS method for large scale optimization - Liu, Nocedal - 1989
266	RB: Numerical Methods for Unconstrained Optimization and Nonlinear Equations - JE, Schnabel - 1983
186	Inexact Newton methods - Dembo, Eisenstat, et al. - 1982
152	Updating quasi-Newton matrices with limited storage - Nocedal - 1980
108	A rapidly convergent descent method for minimization", The Computer Journal 6 - Fletcher, Powell - 1963
101	The conjugate gradient method and trust regions in large scale optimization - Steihaug - 1983
80	A numerically stable dual method for solving strictly convex quadratic programs - Goldfarb, Idnani - 1983
73	Globally convergent inexact Newton methods - Eisenstat, Walker - 1994
71	Variable metric method for minimization - Davidon - 1959
70	Thuente, Line search algorithms with guaranteed sufficient decrease - Moré, J - 1994
66	Recent developments in algorithms and software for trust region methods - Moré - 1983
66	Restart procedures for the conjugate gradient method - Powell - 1977
55	A survey of truncated-Newton methods - Nash - 2000
53	Global convergence properties of conjugate gradient methods for optimization - JC, Nocedal - 1992
50	Convergence conditions for ascent methods - Wolfe - 1969
49	Note sur la convergence de méthodes de directions conjuguées - Polak, Ribière - 1969
47	A method for solving a convex programming problem with convergence rate 1/k 2 . Soviet mathematics - Nesterov - 1983
43	Mor' e A characterization of superlinear convergence and its application to quasi-Newton methods - Dennis, J - 1974
37	Conjugate gradient methods with inexact searches - Shanno - 1978
36	A tool for the analysis of quasi-Newton methods with application to unconstrained minimization - Byrd, Nocedal - 1989
35	Mor' e On the local and superlinear convergence of quasi-Newton methods - Broyden, Dennis, et al. - 1973
35	1989]: Some numerical experiments with variable storage quasi-Newton algorithms - GILBERT, LEMARECHAL
30	Linear and Non Linear Programming,” 2nd Edition - Luenberger - 1989
30	Convergence Conditions for Ascent Methods II: Some Corrections - WOLFE
29	Some global convergence properties of a variable metric algorithm for minimization without exact line searches - Powell - 1976
28	Nonconvex minimization calculations and the conjugate gradient method - Powell - 1984
26	A Derivation of Conjugate Gradients - Beale - 1972
24	Descent property and global convergence of the Fletcher-Reeves method with inexact line search - Al-Baali - 1985
24	A class of nonmonotone stabilization methods in unconstrained optimization, Numerische Mathematik 59 - Grippo, Lampariello, et al. - 1991
24	A discrete Newton algorithm for minimizing a function of many variables - O’LEARY - 1982
23	On a successive transformation of probability distribution and its application to the analysis of the optimum gradient method - Akaike - 1959
22	Global convergence of a class of quasi-Newton methods on convex problems - Byrd, Nocedal, et al. - 1987
19	Algorithm 500: Minimization of Unconstrained Multivariate Functions [E4 - Shanno, Phua - 1980
17	A theoretical and experimental study of the symmetric rank-one update - Khalfan, Byrd, et al. - 1993
17	On the global convergence of trust region algorithms for unconstrained minimization - Powell - 1984
15	Sizing and least-change secant methods - Wolkowicz - 1993
14	Conjugate-Gradient Methods for Large-Scale Nonlinear Optimization - GILL, MURRAY - 1979
13	QN-like variable storage conjugate gradients - BUCKLEY, LENto - 1983
13	Numerical experiments with partially separable optimization problems - GRIEWANK, Toint - 1983
13	Perspectives on self-scaling variable metric algorithms - Oren - 1974
13	On the convergence of the variable metric algorithm - Powell - 1971
12	Linear convergence of the conjugate gradient method - CROWDER, WOLFE - 1972
12	Optimally Conditioned Optimization Algorithms Without Line Searches - Davidon - 1975
12	Quasi-Newton methods, motivation and theory - Moré - 1977
12	Local convergence theory of inexact Newton methods based on structured least change updates - Mart'inez - 1990
11	Rate of convergence of several conjugate gradient algorithms - Cohen - 1972
11	A view of unconstrained optimization - Schnabel - 1989
11	A global convergence theory for arbitrary norm trust region methods for nonlinear equations - Hallabi, Tapia - 1993