Abstract. Direct search methods are best known as unconstrained optimization techniques that do not explicitly use derivatives. Direct search methods were formally proposed and widely applied in the 1960s but fell out of favor with the mathematical optimization community by the early 1970s because they lacked coherent mathematical analysis. Nonetheless, users remained loyal to these methods, most of which were easy to program, some of which were reliable. In the past fifteen years, these methods have seen a revival due, in part, to the appearance of mathematical analysis, as well as to interest in parallel and distributed computing. This review begins by briefly summarizing the history of direct search methods and considering the special properties of problems for which they are well suited. Our focus then turns to a broad class of methods for which we provide a unifying framework that lends itself to a variety of convergence results. The underlying principles allow generalization to handle bound constraints and linear constraints. We also discuss extensions to problems with nonlinear constraints.

Abstract not found

Almost by definition, well-tuned digital circuits have a large number of equally critical paths, which form a so-called "wall" in the slack histogram. However, by the time the design has been through manufacturing, many uncertainties cause these carefully aligned delays to spread out. Inaccuracies in parasitic predictions, clock slew, model-to-hardware correlation, static timing assumptions and manufacturing variations all cause the performance to vary from prediction. Simple statistical principles tell us that the variation of the limiting slack is larger when the height of the wall is greater. Although the wall may be the optimum solution if the static timing predictions were perfect, in the presence of uncertainty in timing and manufacturing, it may no longer be the best choice. The application of formal mathematical optimization in transistor sizing increases the height of the wall, thus exacerbating the problem. There is also a practical matter that schematic restructuring downstream in the design methodology is easier to conceive when there are fewer equally critical paths. This paper describes a method that gives formal mathematical optimizers the incentive to avoid the wall of equally critical paths, while giving up as little as possible in nominal performance. Surprisingly, such a formulation reduces the degeneracy of the optimization problem and can render the optimizer more effective. This "uncertainty-aware" mode has been implemented and applied to several high-performance microprocessor macros. Numerical results are included.

As a means to obtain a more accurate description of traffic flows than that provided by the basic model of traffic assignment, there have been suggestions to impose upper bounds on the link flows. This can be done either by introducing explicit link capacities or by employing travel time functions with asymptotes at the upper bounds. Although the latter alternative has the disadvantage of inherent numerical ill-conditioning, the capacitated assignment model has been studied and applied to a limited extent, the main reason being that the solutions can not be characterized by the classical Wardrop equilibrium conditions; they may, however, be characterized as Wardrop equilibria in terms of a welldefined, natural generalized travel cost. The introduction of link capacity side constraints makes the problem computationally more demanding. The availability of efficient algorithms for the basic model of traffic assignment motivates the use of dualization approaches for handling the capacity c...

In this paper, we discuss a spectral quadratic-SDP method for the iterative resolution of fixed-order H2 and H ∞ design problems. These problems can be cast as regular SDP programs with additional nonlinear equality constraints. When the inequalities are absorbed into a Lagrangian function the problem reduces to solving a sequence of SDPs with quadratic objective function for which a spectral SDP method has been developed. Along with a description of the spectral SDP method used to solve the tangent subproblems, we report a number of computational results for validation purposes.

This supplement provides detailed information about the functionalities of the Potters-Wheel toolbox as described in the main text. For further information please use the

For design problems involving computation-intensive analysis or simulation processes, approximation models are usually introduced to reduce computation time. Most approximation-based optimization methods make step-by-step improvements to the approximation model by adjusting the limits of the design variables. In this work, a new approximation-based optimization method for computation-intensive design problems — the adaptive response surface method (ARSM), is presented. The ARSM creates quadratic approximation models for the computation-intensive design objective function in a gradually reduced design space. The ARSM was designed to avoid being trapped by local optimum and to identify the global design optimum with a modest number of objective function evaluations. Extensive tests on the ARSM as a global optimization scheme using benchmark problems, as well as an industrial design application of the method, are presented. Advantages and limitations of the approach are also discussed.

We consider the solution of nonlinear programs in the case where derivatives of the objective function and nonlinear constraints are unavailable. To solve such problems, we propose an adaptation of a method due to Conn, Gould, Sartenaer, and Toint that proceeds by approximately minimizing a succession of linearly constrained augmented Lagrangians. Our modification is to use a derivative-free generating set direct search algorithm to solve the linearly constrained subproblems. The stopping criterion proposed by Conn, Gould, Sartenaer and Toint for the approximate solution of the subproblems requires explicit knowledge of derivatives. Such information is presumed absent in the generating set search method we employ. Instead, we show that stationarity results for linearly constrained generating set search methods provide a derivative-free stopping criterion, based on a step-length control parameter, that is sufficient to preserve the convergence properties of the original augmented Lagrangian algorithm.

Gaussian decomposition of images leads to many promising applications in computer graphics. Gaussian representations can be used for image smoothing, motion analysis, and feature selection for image recognition. Furthermore, image construction from a Gaussian representation is fast, since the Gaussians only need to be added together. The most optimal algorithms [3, 6, 7] minimize the number of Gaussians needed for decomposition, but they involve nonlinear least-squares approximations, e.g. the use of the Marquardt algorithm [10]. This presents a problem, since, in the Marquardt algorithm, enormous amounts of computations are required and the resulting matrices use a lot of space. In this work, a method is offered, which we call the Quickstep method, that substantially reduces the number of computations and the amount of space used. Unlike the Marquardt algorithm, each iteration has linear time complexity in the number of variables and no Jacobian or Hessian matrices are formed. Yet, Quickstep produces optimal results, similar to those produced by the Marquardt algorithm.

Communicated by C. T. Leondes 1This work was supported by PRONEX-CNPq/FAPERJ Grant E-26/171.164/2003- APQ1, FAPESP Grants 03/09169-6 and 01/04597-4, and CNPq. The authors are indebted to Juliano B. Francisco and Yalcin Kaya for their careful reading of the first draft of this paper.