Abstract

: Many different procedures have been proposed for optimization calculations when first derivatives are not available. Further, several researchers have contributed to the subject, including some who wish to prove convergence theorems, and some who wish to make any reduction in the least calculated value of the objective function. There is not even a key idea that can be used as a foundation of a review, except for the problem itself, which is the adjustment of variables so that a function becomes least, where each value of the function is returned by a subroutine for each trial vector of variables. Therefore the paper is a collection of essays on particular strategies and algorithms, in order to consider the advantages, limitations and theory of several techniques. The subjects that are addressed are line search methods, the restriction of vectors of variables to discrete grids, the use of geometric simplices, conjugate direction procedures, trust region algorithms that form linear or...

Citations

6071	Genetic Algorithms - Goldberg - 1989
1128	Practical Optimization - Gill, Murray, et al. - 1981
953	A simplex method for function minimization - NELDER, MEAD - 1965
244	Algorithms for Minimization without Derivatives - Brent - 1973
180	Jeeves, Direct search solution of numerical and statistical problems - Hooke, A - 1961
159	An efficient method for finding the minimum of a function of several variables without calculating derivatives. The Computer Journal - Powell - 1964
104	On the convergence of pattern search algorithms - Torczon - 1997
98	Direct search methods on parallel machines - Dennis, Torczon - 1991
86	An Automatic Method of Finding the Greatest or Least Value of Function - Rosenbrock - 1960
64	Sequential application of simplex designs in optimisation and evolutionary operation - SPENDLEY, HEXT, et al. - 1962
41	Convergence of the Nelder–Mead simplex method to a nonstationary point - McKinnon - 1998
38	A Direct Search Optimization Method That Models the Objective and Constraint Functions by Linear Interpolation,” DAMTP/NA5 - Powell - 1992
22	A Grid Algorithm for Bound Constrained Optimization of Noisy Functions - Elster, Neumaier - 1995
21	Detection and remediation of stagnation in the Nelder–Mead algorithm using a sufficient decrease condition - Kelley - 1999
13	On Search Directions for Minimization Algorithms - Powell - 1973
11	Teukolsky and W.T. Vetterling - Press, Flannery, et al. - 1992
8	Global convergence and stabilization of unconstrained minimization methods without derivatives - Grippo, Lampariello, et al. - 1988
8	On the global convergence of derivative free methods for unconstrained optimization - Lucidi, Sciandrone - 1996
8	Function minimization by interpolation in a data table - Winfield - 1973
5	The automatic computation of maximum likelihood estimates - Smith - 1962
4	On the accelerating property of an algorithm for function minimization without calculating derivatives - Toint, Callier - 1977
3	A new direction set method for unconstrained minimization without evaluating derivatives - Brodlie - 1975
3	Scheinberg and Ph.L. Toint (1997a), "On the convergence of derivative-free methods for unconstrained optimization - Conn, K
3	Scheinberg and Ph.L. Toint (1997b), "Recent progress in unconstrained nonlinear optimization without derivatives - Conn, K