## On the Convergence of Pattern Search Algorithms (0)

Citations: | 149 - 14 self |

### BibTeX

@MISC{Torczon_onthe,

author = {Virginia Torczon},

title = {On the Convergence of Pattern Search Algorithms},

year = {}

}

### Years of Citing Articles

### OpenURL

### Abstract

. We introduce an abstract definition of pattern search methods for solving nonlinear unconstrained optimization problems. Our definition unifies an important collection of optimization methods that neither computenor explicitly approximate derivatives. We exploit our characterization of pattern search methods to establish a global convergence theory that does not enforce a notion of sufficient decrease. Our analysis is possible because the iterates of a pattern search method lie on a scaled, translated integer lattice. This allows us to relax the classical requirements on the acceptance of the step, at the expense of stronger conditions on the form of the step, and still guarantee global convergence. Key words. unconstrained optimization, convergence analysis, direct search methods, globalization strategies, alternating variable search, axial relaxation, local variation, coordinate search, evolutionary operation, pattern search, multidirectional search, downhill simplex search AMS(M...

### Citations

1381 |
A simplex method for function minimization
- Nelder, Mead
- 1965
(Show Context)
Citation Context ...rate; as such, multidirectional search owes much in its conception to its predecessors, the simplex design algorithm of Spendley, Hext, and Himsworth [16] and the simplex algorithm of Nelder and Mead =-=[12]-=-. However, multidirectional search is a different algorithm---particularly from a theoretical standpoint. Convergence for the Spendley, Hext and Himsworth algorithm can be shown only with some modific... |

511 |
Iterative Solution of Nonlinear Equations
- Ortega, Rheinboldt
- 2000
(Show Context)
Citation Context ... line search theory are perhaps most obvious and are discussed in [20]. The outline for the convergence theory follows the outline for global convergence theorems as detailed by Ortega and Rheinboldt =-=[14]-=- and reviewed in the survey by Nocedal [13]: we consider iterations of the form x k+1 = x k + s k where s k 2 \Delta k P k ; the columns of P k determine the search directions and \Delta k serves as a... |

316 |
Numerical methods for unconstrained optimization and nonlinear equations: Prentice-Hall, Englewood Cliffs
- DennisJr, Schnabel
- 1983
(Show Context)
Citation Context ...\Delta k P k ; the columns of P k determine the search directions and \Delta k serves as a step length parameter. In x7.1 we show that pattern search methods are descent methods (as defined in either =-=[8]-=- or [14]) by showing that if x k is not a stationary point of the function then the generalized pattern search method guarantees decrease in the value of the objective function in a finite number of i... |

139 | On the Experimental Attainment of Optimum Conditions - Box, Wilson - 1951 |

135 |
Nonlinear Programming: Analysis and Methods
- Avriel
- 2003
(Show Context)
Citation Context ...ate vectors and c p k denotes the last column of C k . We set ae \Gamma1 = 0 so that ae k\Gamma1 is defined when k = 0. A useful example for working through the logic of the algorithm can be found in =-=[1]-=-, though the presentation and notation differ somewhat from that given here. Algorithm 6. Exploratory Moves Algorithm for Hooke and Jeeves. Given x k , \Delta k , f(x k ), B, and ae k\Gamma1 , set ae ... |

112 | Direct search methods on parallel machines
- Jr, Torczon
- 1991
(Show Context)
Citation Context ... searches based on two-level factorial designs (see [4] and [5]), ffl the original pattern search method of Hooke and Jeeves [10], and ffl the multidirectional search algorithm of Dennis and Torczon (=-=[9]-=- and [19]). We will show that these algorithms satisfy the conditions that define pattern search methods and thus are special cases of the generalized pattern search method presented as Algorithm 1. T... |

94 |
Variable metric method for minimization
- Davidon
- 1959
(Show Context)
Citation Context ...e simplest and most obvious of all the pattern search methods. Davidon describes it concisely in the opening of his belated preface to Argonne National Laboratory Research and Development Report 5990 =-=[7]-=-: Enrico Fermi and Nicholas Metropolis used one of the first digital computers, the Los Alamos Maniac, to determine which values of certain theoretical parameters (phase shifts) best fit experimental ... |

83 | Theory of Algorithms for Unconstrained Optimization
- Nocedal
- 1992
(Show Context)
Citation Context ...s and are discussed in [20]. The outline for the convergence theory follows the outline for global convergence theorems as detailed by Ortega and Rheinboldt [14] and reviewed in the survey by Nocedal =-=[13]-=-: we consider iterations of the form x k+1 = x k + s k where s k 2 \Delta k P k ; the columns of P k determine the search directions and \Delta k serves as a step length parameter. In x7.1 we show tha... |

78 |
Evolutionary operation: A method for increasing industrial productivity
- Box
(Show Context)
Citation Context ...d, individually, to such classical algorithms as coordinate search, with fixed step sizes, variants of response surface methodology, first developed by Box and Wilson [4] and later popularized by Box =-=[2, 3]-=-, and the original pattern search algorithm of Hooke and Jeeves [10]. The challenge was to develop an abstraction that both allowed for a general convergence theory and explained why such algorithms, ... |

78 |
Sequential application of simplex design in optimization and evolutionary operations
- Spendley, Hext, et al.
- 1962
(Show Context)
Citation Context ...n implicit information about the gradient; they do not construct local quadratic models of the function. The only natural notion of a full step is that given by \Delta k . As has long been recognized =-=[16]-=-, this means that pattern search methods do not enjoy fast local convergence properties. When using these methods one trades speed of convergence for robustness and wider applicability. One other poin... |

76 |
Computational Methods in Optimization: A Unified Approach
- Polak
- 1971
(Show Context)
Citation Context ...Rice University, Houston, TX 77251-1892. 2 VIRGINIA TORCZON depends solely on the sequence of updates that have been applied to the step length control parameter. Despite isolated convergence results =-=[4, 11, 16]-=- for certain individual pattern search methods, a general theory of convergence for the class of such methods remained elusive for some time. The standard convergence theory for line search and trust ... |

63 |
Multi-Directional Search: A Direct Search Algorithm for Parallel Machines
- Torczon
- 1989
(Show Context)
Citation Context ...s based on two-level factorial designs (see [4] and [5]), ffl the original pattern search method of Hooke and Jeeves [10], and ffl the multidirectional search algorithm of Dennis and Torczon ([9] and =-=[19]-=-). We will show that these algorithms satisfy the conditions that define pattern search methods and thus are special cases of the generalized pattern search method presented as Algorithm 1. Then we ca... |

63 | On the convergence of the multidirectional search algorithm
- Torczon
- 1991
(Show Context)
Citation Context ...Rice University, Houston, TX 77251-1892. 2 VIRGINIA TORCZON depends solely on the sequence of updates that have been applied to the step length control parameter. Despite isolated convergence results =-=[4, 11, 16]-=- for certain individual pattern search methods, a general theory of convergence for the class of such methods remained elusive for some time. The standard convergence theory for line search and trust ... |

49 |
Convergence properties of a class of minimization algorithms
- Powell
- 1975
(Show Context)
Citation Context ...t region. It is worth comparing the global first-order stationary point convergence results for trust region methods and generalized pattern search methods. Roughly speaking, the result due to Powell =-=[15]-=- says that lim inf k!+1 krf(x k )k = 0 assuming that rf is continuous and the Hessian (or its approximation) remains uniformly bounded. Thomas [18] assumes, in addition, that rf is uniformly continuou... |

26 |
e, Recent developments in algorithms and software for trust region methods
- Mor'
- 1983
(Show Context)
Citation Context ...f of Theorem 3.7 is almost identical to that of an equivalent result for trust region methods that was first given by Thomas [18] and which is included, in a more general form, in the survey by Mor'e =-=[11]-=-. One final note: the hypotheses of Theorem 3.7 suggest that in the absence of any explicit higher-order information about the function to be minimized, it makes sense to terminate a generalized patte... |

24 | PDS: Direct search methods for unconstrained optimization on either sequential or parallel machines - Torczon - 1992 |

16 | Discrete search methods in Numerical Methods for Unconstrained Optimization - Swann - 1972 |

13 |
Sequential estimation techniques for quasi-Newton algorithms
- THOMAS
- 1975
(Show Context)
Citation Context ...or any j, 0 ! j ! ", we have a contradiction (e.g., try j = " 4 ). The proof of Theorem 3.7 is almost identical to that of an equivalent result for trust region methods that was first given =-=by Thomas [18]-=- and which is included, in a more general form, in the survey by Mor'e [11]. One final note: the hypotheses of Theorem 3.7 suggest that in the absence of any explicit higher-order information about th... |

13 | Direct search" solution of numerical and statistical problems - Hooke, Jeeves - 1961 |

11 |
The exploration and exploitation of Response surfaces; some general, considerations and examples
- Box
- 1954
(Show Context)
Citation Context ...d, individually, to such classical algorithms as coordinate search, with fixed step sizes, variants of response surface methodology, first developed by Box and Wilson [4] and later popularized by Box =-=[2, 3]-=-, and the original pattern search algorithm of Hooke and Jeeves [10]. The challenge was to develop an abstraction that both allowed for a general convergence theory and explained why such algorithms, ... |

6 |
Jeeves, "Direct search" solution of numerical and statistical problems
- Hooke, A
- 1961
(Show Context)
Citation Context ...th fixed step sizes, variants of response surface methodology, first developed by Box and Wilson [4] and later popularized by Box [2, 3], and the original pattern search algorithm of Hooke and Jeeves =-=[10]-=-. The challenge was to develop an abstraction that both allowed for a general convergence theory and explained why such algorithms, often viewed as disparate direct search methods, could be analyzed u... |

6 | Positive basis and a class of direct search techniques - Wen-ci - 1979 |

4 | ea, Optimisation : th'eorie et algorithmes - C' - 1971 |

3 |
Direct search methods for unconstrained optimization on either sequential or parallel machines
- PDS
- 1992
(Show Context)
Citation Context ... when \Delta k is less than some reasonably small tolerance. In fact, this is a common stopping condition for algorithms of this sort and the one implemented for the multidirectional search algorithm =-=[21]-=-. 4. Parallels with Line Search and Model Trust Region Theory. The global convergence analysis we have just presented for pattern search methods shares similarities with the global convergence analysi... |