Results 1  10
of
183
Optimization by direct search: New perspectives on some classical and modern methods
 SIAM Review
, 2003
"... Abstract. Direct search methods are best known as unconstrained optimization techniques that do not explicitly use derivatives. Direct search methods were formally proposed and widely applied in the 1960s but fell out of favor with the mathematical optimization community by the early 1970s because t ..."
Abstract

Cited by 125 (14 self)
 Add to MetaCart
Abstract. Direct search methods are best known as unconstrained optimization techniques that do not explicitly use derivatives. Direct search methods were formally proposed and widely applied in the 1960s but fell out of favor with the mathematical optimization community by the early 1970s because they lacked coherent mathematical analysis. Nonetheless, users remained loyal to these methods, most of which were easy to program, some of which were reliable. In the past fifteen years, these methods have seen a revival due, in part, to the appearance of mathematical analysis, as well as to interest in parallel and distributed computing. This review begins by briefly summarizing the history of direct search methods and considering the special properties of problems for which they are well suited. Our focus then turns to a broad class of methods for which we provide a unifying framework that lends itself to a variety of convergence results. The underlying principles allow generalization to handle bound constraints and linear constraints. We also discuss extensions to problems with nonlinear constraints.
Global optimization by multilevel coordinate search
 J. Global Optimization
, 1999
"... Abstract. Inspired by a method by Jones et al. (1993), we present a global optimization algorithm based on multilevel coordinate search. It is guaranteed to converge if the function is continuous in the neighborhood of a global minimizer. By starting a local search from certain good points, an impro ..."
Abstract

Cited by 73 (11 self)
 Add to MetaCart
Abstract. Inspired by a method by Jones et al. (1993), we present a global optimization algorithm based on multilevel coordinate search. It is guaranteed to converge if the function is continuous in the neighborhood of a global minimizer. By starting a local search from certain good points, an improved convergence result is obtained. We discuss implementation details and give some numerical results.
A Radial Basis Function Method for Global Optimization
 JOURNAL OF GLOBAL OPTIMIZATION
, 1999
"... We introduce a method that aims to find the global minimum of a continuous nonconvex function on a compact subset of R^d. It is assumed that function evaluations are expensive and that no additional information is available. Radial basis function interpolation is used to define a utility function. T ..."
Abstract

Cited by 49 (1 self)
 Add to MetaCart
We introduce a method that aims to find the global minimum of a continuous nonconvex function on a compact subset of R^d. It is assumed that function evaluations are expensive and that no additional information is available. Radial basis function interpolation is used to define a utility function. The maximizer of this function is the next point where the objective function is evaluated. We show that, for most types of radial basis functions that are considered in this paper, convergence can be achieved without further assumptions on the objective function. Besides, it turns out that our method is closely related to a statistical global optimization method, the Palgorithm. A general framework for both methods is presented. Finally, a few numerical examples show that on the set of DixonSzego test functions our method yields favourable results in comparison to other global optimization methods.
Subdivision Direction Selection In Interval Methods For Global Optimization
 SIAM J. Numer. Anal
, 1997
"... . The role of the interval subdivision selection rule is investigated in branchandbound algorithms for global optimization. The class of rules that allow convergence for the model algorithm is characterized, and it is shown that the four rules investigated satisfy the conditions of convergence. A ..."
Abstract

Cited by 46 (18 self)
 Add to MetaCart
. The role of the interval subdivision selection rule is investigated in branchandbound algorithms for global optimization. The class of rules that allow convergence for the model algorithm is characterized, and it is shown that the four rules investigated satisfy the conditions of convergence. A numerical study with a wide spectrum of test problems indicates that there are substantial differences between the rules in terms of the required CPU time, the number of function and derivative evaluations and space complexity, and two rules can provide substantial improvements in efficiency. Key words. global optimization, interval arithmetic, interval subdivision AMS subject classifications. 65K05, 90C30 Abbreviated title: Subdivision directions in interval methods. 1. Introduction. Interval subdivision methods for global optimization [7, 21] aim at providing reliable solutions to global optimization problems min x2X f(x) (1) where the objective function f : IR n ! IR is continuo...
Flexibility and Efficiency Enhancements for Constrained Global Design Optimization with Kriging Approximations
, 2002
"... ..."
Simulated Annealing Algorithms For Continuous Global Optimization
, 2000
"... INTRODUCTION In this paper we consider Simulated Annealing algorithms (SA in what follows) applied to continuous global optimization problems, i.e. problems with the following form f = min x2X f(x); (1.1) where X ` ! n is a continuous domain, often assumed to be compact, which, combined with ..."
Abstract

Cited by 30 (1 self)
 Add to MetaCart
INTRODUCTION In this paper we consider Simulated Annealing algorithms (SA in what follows) applied to continuous global optimization problems, i.e. problems with the following form f = min x2X f(x); (1.1) where X ` ! n is a continuous domain, often assumed to be compact, which, combined with the continuity or lower semicontinuity of f , guarantees the existence of the minimum value f . SA algorithms are based on an analogy with a physical phenomenon: while at high temperatures the molecules in a liquid move freely, if the temperature is slowly decreased the thermal mobility of the molecules is lost and they form a pure crystal which also corresponds to a state of minimum energy. If the temperature is decreased too quickly (the so called quenching) a liquid metal rather ends up in a polycrystalline or amorphous state with
On the Selection of Subdivision Directions in Interval BranchandBound Methods for Global Optimization
 J. Global Optimization
, 1995
"... . This paper investigates the influence of the interval subdivision selection rule on the convergence of interval branchandbound algorithms for global optimization. For the class of rules that allows convergence, we study the effects of the rules on a model algorithm with special list ordering. Fo ..."
Abstract

Cited by 30 (13 self)
 Add to MetaCart
. This paper investigates the influence of the interval subdivision selection rule on the convergence of interval branchandbound algorithms for global optimization. For the class of rules that allows convergence, we study the effects of the rules on a model algorithm with special list ordering. Four different rules are investigated in theory and in practice. A wide spectrum of test problems is used for numerical tests indicating that there are substantial differences between the rules with respect to the required CPU time, the number of function and derivative evaluations, and the necessary storage space. Two rules can provide considerable improvements in efficiency for our model algorithm. Keywords: Global optimization, interval arithmetic, branchandbound, interval subdivision 1. Introduction The investigated class of interval branchandbound methods for global optimization [7], [8], [19] addresses the problem of finding guaranteed and reliable solutions of global optimization...
A tutorial on Bayesian optimization of expensive cost functions, withapplicationtoactiveusermodeling andhierarchical reinforcement learning
, 2009
"... We present a tutorial on Bayesian optimization, a method of finding the maximum of expensive cost functions. Bayesian optimization employs the Bayesian technique of setting a prior over the objective function and combining it with evidence to get a posterior function. This permits a utilitybased se ..."
Abstract

Cited by 28 (2 self)
 Add to MetaCart
We present a tutorial on Bayesian optimization, a method of finding the maximum of expensive cost functions. Bayesian optimization employs the Bayesian technique of setting a prior over the objective function and combining it with evidence to get a posterior function. This permits a utilitybased selection of the next observation to make on the objective function, which must take into account both exploration (sampling from areas of high uncertainty) and exploitation (sampling areas likely to offer improvement over the current best observation). We also present two detailed extensions of Bayesian optimization, with experiments—active user modelling with preferences, and hierarchical reinforcement learning— and a discussion of the pros and cons of Bayesian optimization based on our experiences. 1
A LocallyBiased Form Of The Direct Algorithm
 Journal of Global Optimization
, 2001
"... . In this paper we propose a form of the DIRECT algorithm that is strongly biased toward local search. This form should do well for small problems with a single global minimizer and only a few local minimizers. We motivate our formulation with some results on how the original formulation of the DIRE ..."
Abstract

Cited by 27 (4 self)
 Add to MetaCart
. In this paper we propose a form of the DIRECT algorithm that is strongly biased toward local search. This form should do well for small problems with a single global minimizer and only a few local minimizers. We motivate our formulation with some results on how the original formulation of the DIRECT algorithm clusters its search near a global minimizer. We report on the performance of our algorithm on a suite of test problems and observe that the algorithm performs particularly well when termination is based on a budget of function evaluations. Key words. DIRECT, local clustering, local bias 1. Introduction. The DIRECT (DIviding RECTangles) algorithm [13, 14] is a pattern search method (in the sense of [17]) that balances local and global search in a attempt to efficiently find a global optimizer. Other deterministic sampling methods, such as implicit filtering [9, 15], MDS [6], HookeJeeves [10], or NelderMead [16], drive an approximate gradient to zero and are not designed for g...