Results 1  10
of
16
An Implicit Filtering Algorithm For Optimization Of Functions With Many Local Minima
 SIAM J. Optim
, 1995
"... . In this paper we describe and analyze an algorithm for certain box constrained optimization problems that may have several local minima. A paradigm for these problems is one in which the function to be minimized is the sum of a simple function, such as a convex quadratic, and high frequency, low a ..."
Abstract

Cited by 53 (16 self)
 Add to MetaCart
. In this paper we describe and analyze an algorithm for certain box constrained optimization problems that may have several local minima. A paradigm for these problems is one in which the function to be minimized is the sum of a simple function, such as a convex quadratic, and high frequency, low amplitude terms which cause local minima away from the global minimum of the simple function. Our method is gradient based and therefore the performance can be improved by use of quasiNewton methods. Key words. filtering, projected gradient algorithm, quasiNewton method AMS(MOS) subject classifications. 65H10, 65K05, 65K10 1. Introduction. In this paper we describe and analyze an algorithm for bound constrained optimization problems that may have several local minima. The type of problem we have in mind is one in which the function to be minimized is the sum of a simple function, such as a convex quadratic, and high frequency, low amplitude terms which cause the local minima. Of particul...
Detection And Remediation Of Stagnation In The NelderMead Algorithm Using A Sufficient Decrease Condition
 SIAM J. OPTIM
, 1997
"... The NelderMead algorithm can stagnate and converge to a nonoptimal point, even for very simple problems. In this note we propose a test for sufficient decrease which, if passed for the entire iteration, will guarantee convergence of the NelderMead iteration to a stationary point if the objective ..."
Abstract

Cited by 31 (1 self)
 Add to MetaCart
The NelderMead algorithm can stagnate and converge to a nonoptimal point, even for very simple problems. In this note we propose a test for sufficient decrease which, if passed for the entire iteration, will guarantee convergence of the NelderMead iteration to a stationary point if the objective function is smooth. Failure of this condition is an indicator of potential stagnation. As a remedy we propose a new step, which we call an oriented restart, which reinitializes the simplex to a smaller one with orthogonal edges which contains an approximate steepest descent step from the current best point. We also give results that apply when objective function is a lowamplitude perturbation of a smooth function. We illustrate our results with some numerical examples.
Superlinear Convergence And Implicit Filtering
, 1999
"... . In this note we show how the implicit filtering algorithm can be coupled with the BFGS quasiNewton update to obtain a superlinearly convergent iteration if the noise in the objective function decays sufficiently rapidly as the optimal point is approached. We show how known theory for the noisefr ..."
Abstract

Cited by 22 (3 self)
 Add to MetaCart
. In this note we show how the implicit filtering algorithm can be coupled with the BFGS quasiNewton update to obtain a superlinearly convergent iteration if the noise in the objective function decays sufficiently rapidly as the optimal point is approached. We show how known theory for the noisefree case can be extended and thereby provide a partial explanation for the good performance of quasiNewton methods when coupled with implicit filtering. Key words. noisy optimization, implicit filtering, BFGS algorithm, superlinear convergence AMS subject classifications. 65K05, 65K10, 90C30 1. Introduction. In this paper we examine the local and global convergence behavior of the combination of the BFGS [4], [20], [17], [23] quasiNewton method with the implicit filtering algorithm. The resulting method is intended to minimize smooth functions that are perturbed with lowamplitude noise. Our results, which extend those of [5], [15], and [6], show that if the amplitude of the noise decays ...
The Simplex Gradient and Noisy Optimization Problems
 in Computational Methods in Optimal Design and Control
, 1998
"... this paper we consider objective functions that are perturbations of simple, smooth functions. The surface in on the left in Figure 1, taken from [24], and the graph on the right illustrate this type of problem. Figure 1: Optimization Landscapes 0 5 10 15 20 25 0 5 10 15 20 25 80 60 40 20 0 20 0 ..."
Abstract

Cited by 19 (4 self)
 Add to MetaCart
this paper we consider objective functions that are perturbations of simple, smooth functions. The surface in on the left in Figure 1, taken from [24], and the graph on the right illustrate this type of problem. Figure 1: Optimization Landscapes 0 5 10 15 20 25 0 5 10 15 20 25 80 60 40 20 0 20 0.5 1.5 2.5 3.5 4.5 21.51 0.5 0.5 1 1.5 2 The perturbations may be results of discontinuities or nonsmoth effects in the underlying models, randomness in the function evaluation, or experimental or measurement errors. Conventional gradientbased methods will be trapped in local minima even if the noise is smooth. Many classes of methods for noisy optimization problems are based on function information computed on sequences of simplices. The NelderMead, [18], multidirectional search, [8], [21], and implicit filtering, [12], methods are three examples. The performance of such methods can be explained in terms of the difference approximation of the gradient that is implicit in the function evaluations they perform.
Algorithms for Noisy Problems in Gas Transmission Pipeline Optimization
, 2000
"... In this paper we describe some algorithms for noisy optimization in the context of problems from the gas transmission industry. The algorithms are implicit filtering, DIRECT, and a new hybrid of these methods, which uses DIRECT to find an initial iterate for implicit filtering. We report on numerica ..."
Abstract

Cited by 18 (5 self)
 Add to MetaCart
In this paper we describe some algorithms for noisy optimization in the context of problems from the gas transmission industry. The algorithms are implicit filtering, DIRECT, and a new hybrid of these methods, which uses DIRECT to find an initial iterate for implicit filtering. We report on numerical results that illustrate the performance of the methods.
Optimization Of Automotive Valve Train Components With Implict Filtering
 Optimization and Engineering
, 1998
"... . In this paper we show how the implicit filtering algorithm can be parallelized and applied to problems in parameter identification and optimization in automotive valve train design. We extend our previous work by using a more refined model of the valve train and exploiting parallelism in a new way ..."
Abstract

Cited by 10 (4 self)
 Add to MetaCart
. In this paper we show how the implicit filtering algorithm can be parallelized and applied to problems in parameter identification and optimization in automotive valve train design. We extend our previous work by using a more refined model of the valve train and exploiting parallelism in a new way. We apply the parameter identification results to obtain optimal profiles for camshaft lobes. Key words. Noisy Optimization, Implicit Filtering, Mechanical Systems, Automotive Valve Trains AMS subject classifications. 65K05, 65K10, 65L05, 65Y05 1. Introduciton. In this paper we report on a parallel implementation of the implicit filtering [17], [19] algorithm and its application to problems in parameter identification and optimization in automotive valve train design. We extend our previous work [11], [10] on parameter identification by using a more refined model of the valve train and exploiting parallelism in a new way. We then apply the parameter identification results to obtain optim...
Comparison of DerivativeFree Optimization Methods for Groundwater Supply and Hydraulic Capture Community Problems
 ADVANCES IN WATER RESOURCES
, 2008
"... Management decisions involving groundwater supply and remediation often rely on optimization techniques to determine an effective strategy. We introduce several derivativefree sampling methods for solving constrained optimization problems that have not yet been considered in this field, and we incl ..."
Abstract

Cited by 8 (4 self)
 Add to MetaCart
Management decisions involving groundwater supply and remediation often rely on optimization techniques to determine an effective strategy. We introduce several derivativefree sampling methods for solving constrained optimization problems that have not yet been considered in this field, and we include a genetic algorithm for completeness. Two welldocumented community problems are used for illustration purposes: a groundwater supply problem and a hydraulic capture problem. The community problems were found to be challenging applications due to the objective functions being nonsmooth, nonlinear, and having many local minima. Because the results were found to be sensitive to initial iterates for some methods, guidance is provided in selecting initial iterates for these problems that improve the likelihood Preprint submitted to Elsevier 14 January 2008of achieving significant reductions in the objective function to be minimized. In addition, we suggest some potentially fruitful areas for future research.
Use of an Implicit Filtering Algorithm for Mechanical . . .
"... Optimal design of highspeed valve trains requires the use of an accurate analytical model. While the governing differential equations are important, the coefficients (or parameters) used in these equations are equally as important. Since many of the parameters used in valve train models are difficu ..."
Abstract

Cited by 6 (6 self)
 Add to MetaCart
Optimal design of highspeed valve trains requires the use of an accurate analytical model. While the governing differential equations are important, the coefficients (or parameters) used in these equations are equally as important. Since many of the parameters used in valve train models are difficult to measure directly, parameter identification based on experimental data is required to assure model accuracy. This paper addresses the parameter identification problem for a valve train model, formulating a scalar cost function which represents the difference in measured and predicted system response. Minimizaton of this cost function yields the 10 unknown system parameters. As the cost function has many local minima, a global optimization scheme must be employed. An implicit filtering algorithm is implemented which applies a scale reduction scheme in conjunction with a gradient projection algorithm to avoid becomming trapped in local minima and thus produces near global minima of the co...
Implicit Filtering And Optimal Design Problems
, 1994
"... Implicit filtering is a form of the gradient projection method of Bertsekas in which the stepsize in a difference approximation of the gradient is changed as the iteration progresses. In this way the algorithm is able to avoid certain types of local minima and in some cases find accurate approximati ..."
Abstract

Cited by 4 (2 self)
 Add to MetaCart
Implicit filtering is a form of the gradient projection method of Bertsekas in which the stepsize in a difference approximation of the gradient is changed as the iteration progresses. In this way the algorithm is able to avoid certain types of local minima and in some cases find accurate approximations to the global minimum. The algorithm is particularly effective in avoiding local minima that are caused by highfrequency lowamplitude terms in the objective function. In this report we will discuss the algorithm and its theoretical properties. We will also present applications for modeling of subsurface contaminant transport and highfield magnet design.