Results 1 
5 of
5
Convergence Properties of the NelderMead Simplex Method in Low Dimensions
 SIAM Journal of Optimization
, 1998
"... Abstract. The Nelder–Mead simplex algorithm, first published in 1965, is an enormously popular direct search method for multidimensional unconstrained minimization. Despite its widespread use, essentially no theoretical results have been proved explicitly for the Nelder–Mead algorithm. This paper pr ..."
Abstract

Cited by 248 (3 self)
 Add to MetaCart
Abstract. The Nelder–Mead simplex algorithm, first published in 1965, is an enormously popular direct search method for multidimensional unconstrained minimization. Despite its widespread use, essentially no theoretical results have been proved explicitly for the Nelder–Mead algorithm. This paper presents convergence properties of the Nelder–Mead algorithm applied to strictly convex functions in dimensions 1 and 2. We prove convergence to a minimizer for dimension 1, and various limited convergence results for dimension 2. A counterexample of McKinnon gives a family of strictly convex functions in two dimensions and a set of initial conditions for which the Nelder–Mead algorithm converges to a nonminimizer. It is not yet known whether the Nelder–Mead method can be proved to converge to a minimizer for a more specialized class of convex functions in two dimensions. Key words. direct search methods, Nelder–Mead simplex methods, nonderivative optimization AMS subject classifications. 49D30, 65K05
Optimization by direct search: New perspectives on some classical and modern methods
 SIAM Review
, 2003
"... Abstract. Direct search methods are best known as unconstrained optimization techniques that do not explicitly use derivatives. Direct search methods were formally proposed and widely applied in the 1960s but fell out of favor with the mathematical optimization community by the early 1970s because t ..."
Abstract

Cited by 126 (14 self)
 Add to MetaCart
Abstract. Direct search methods are best known as unconstrained optimization techniques that do not explicitly use derivatives. Direct search methods were formally proposed and widely applied in the 1960s but fell out of favor with the mathematical optimization community by the early 1970s because they lacked coherent mathematical analysis. Nonetheless, users remained loyal to these methods, most of which were easy to program, some of which were reliable. In the past fifteen years, these methods have seen a revival due, in part, to the appearance of mathematical analysis, as well as to interest in parallel and distributed computing. This review begins by briefly summarizing the history of direct search methods and considering the special properties of problems for which they are well suited. Our focus then turns to a broad class of methods for which we provide a unifying framework that lends itself to a variety of convergence results. The underlying principles allow generalization to handle bound constraints and linear constraints. We also discuss extensions to problems with nonlinear constraints.
Direct Search Methods: Once Scorned, Now Respectable
 In Numerical Analysis 1995 (Proceedings of the 1995 Dundee Biennial Conference in Numerical Analysis
, 1995
"... The need to optimize a function whose derivatives are unknown or nonexistent arises in many contexts, particularly in realworld applications. Various direct search methods, most notably the NelderMead `simplex' method, were proposed in the early 1960s for such problems, and have been enormously p ..."
Abstract

Cited by 83 (3 self)
 Add to MetaCart
The need to optimize a function whose derivatives are unknown or nonexistent arises in many contexts, particularly in realworld applications. Various direct search methods, most notably the NelderMead `simplex' method, were proposed in the early 1960s for such problems, and have been enormously popular with practitioners ever since. Nonetheless, for more than twenty years these methods were typically dismissed or ignored in the mainstream optimization literature, primarily because of the lack of rigorous convergence results. Since 1989, however, direct search methods have been rejuvenated and made respectable. This paper summarizes the history of direct search methods, with special emphasis on the NelderMead method, and describes recent work in this area. This paper is based on a plenary talk given at the Biennial Dundee Conference on Numerical Analysis, Dundee, Scotland, 1995. 1. Introduction Unconstrained optimizationthe problem of minimizing a nonlinear function f(x) for x 2...
Convergence of the NelderMead simplex method to a nonstationary point
 SIAM J. Optim
, 1996
"... . This paper analyses the behaviour of the NelderMead simplex method for a family of examples which cause the method to converge to a nonstationary point. All the examples use continuous functions of two variables. The family of functions contains strictly convex functions with up to three continu ..."
Abstract

Cited by 53 (0 self)
 Add to MetaCart
. This paper analyses the behaviour of the NelderMead simplex method for a family of examples which cause the method to converge to a nonstationary point. All the examples use continuous functions of two variables. The family of functions contains strictly convex functions with up to three continuous derivatives. In all the examples the method repeatedly applies the inside contraction step with the best vertex remaining fixed. The simplices tend to a straight line which is orthogonal to the steepest descent direction. It is shown that this behaviour cannot occur for functions with more than three continuous derivatives. The stability of the examples is analysed. Key words. NelderMead method, direct search, simplex, unconstrained minimization AMS subject classifications. 65K05 1. Introduction. Direct search methods are very widely used in chemical engineering, chemistry and medicine. They are a class of optimization methods which are easy to program, do not require derivatives and a...
FortifiedDescent Simplicial Search Method: A General Approach
 SIAM J. Optim
, 1995
"... We propose a new simplexbased direct search method for unconstrained minimization of a realvalued function f of n variables. As in other methods of this kind, the intent is to iteratively improve an ndimensional simplex through certain reflection/expansion/contraction steps. The method has three n ..."
Abstract

Cited by 15 (1 self)
 Add to MetaCart
We propose a new simplexbased direct search method for unconstrained minimization of a realvalued function f of n variables. As in other methods of this kind, the intent is to iteratively improve an ndimensional simplex through certain reflection/expansion/contraction steps. The method has three novel features. First, a userchosen integer m k specifies the number of "good" vertices to be retained in constructing the initial trial simplicesreflected, then either expanded or contractedat iteration k. Second, a trial simplex is accepted only when it satisfies the criteria of fortified descent, which are stronger than the criterion of strict descent used in most direct search methods. Third, the number of additional function evaluations needed to check a trial reflected/expanded simplex for fortified descent can be controlled. If one of the initial trial simplices satisfies the fortified descent criteria, it is accepted as the new simplex; otherwise, the simplex is shrunk a fracti...