this paper has appeared in

this article I will attempt to review the most recent advances in the theory of unconstrained optimization, and will also describe some important open questions. Before doing so, I should point out that the value of the theory of optimization is not limited to its capacity for explaining the behavior of the most widely used techniques. The question

Abstract not found

. In this note we show how the implicit filtering algorithm can be coupled with the BFGS quasi-Newton update to obtain a superlinearly convergent iteration if the noise in the objective function decays sufficiently rapidly as the optimal point is approached. We show how known theory for the noise-free case can be extended and thereby provide a partial explanation for the good performance of quasi-Newton methods when coupled with implicit filtering. Key words. noisy optimization, implicit filtering, BFGS algorithm, superlinear convergence AMS subject classifications. 65K05, 65K10, 90C30 1. Introduction. In this paper we examine the local and global convergence behavior of the combination of the BFGS [4], [20], [17], [23] quasi-Newton method with the implicit filtering algorithm. The resulting method is intended to minimize smooth functions that are perturbed with low-amplitude noise. Our results, which extend those of [5], [15], and [6], show that if the amplitude of the noise decays ...

: We consider conceptual optimization methods combining two ideas: the MoreauYosida regularization in convex analysis, and quasi-Newton approximations of smooth functions. We outline several approaches based on this combination, and establish their global convergence. Then we study theoretically the local convergence properties of one of these approaches, which uses quasi-Newton updates of the objective function itself. Also, we obtain a globally and superlinearly convergent BFGS proximal method. At each step of our study, we single out the assumptions that are useful to derive the result concerned. Key-words: Bundle methods, convex optimization, global and superlinear convergence, mathematical programming, proximal point, quasi-Newton algorithms, variable metric. (R'esum'e : tsvp) Unite de recherche INRIA Rocquencourt Domaine de Voluceau, Rocquencourt, BP 105, 78153 LE CHESNAY Cedex (France) Telephone : (33 1) 39 63 55 11 -- Telecopie : (33 1) 39 63 53 30 Une famille de m'ethodes d...

In this paper the development, convergence theory and numerical testing of a class of gradient unconstrained minimization algorithms with adaptive stepsize are presented. The proposed class comprises four algorithms: the first two incorporate techniques for the adaptation of a common stepsize for all coordinate directions and the other two allow an individual adaptive stepsize along each coordinate direction. All the algorithms are computationally efficient and possess interesting convergence properties utilizing estimates of the Lipschitz constant that are obtained without additional function or gradient evaluations. The algorithms have been implemented and tested on some well-known test cases as well as on real-life artificial neural network applications and the results have been very satisfactory.

Abstract. This paper reviews the development of different versions of nonlinear conjugate gradient methods, with special attention given to global convergence properties.

We propose a BFGS primal-dual interior point method for minimizing a convex function on a convex set defined by equality and inequality constraints. The algorithm generates feasible iterates and consists in computing approximate solutions of the optimality conditions perturbed by a sequence of positive parameters µ converging to zero. We prove that it converges q-superlinearly for each fixed µ. We also show that it is globally convergent to the analytic center of the primal-dual optimalset when µ tends to 0 and strict complementarity holds.

In previous work, the authors provided a foundation for the theory of variable metric proximal point algorithms for general monotone operators on a Hilbert space. In particular, they develop conditions for the global, linear, and super--linear convergence of their proposed algorithm. This paper focuses attention on two matrix secant updating strategies for the finite dimensional case. These are the Broyden and BFGS updates. The BFGS update is considered for application in the symmetric case, e.g., convex programming applications, while the Broyden update can be applied to general monotone operators. Subject to the linear convergence of the iterates and a quadratic growth condition on the inverse of the operator at the solution, both updates are shown to yield the super--linear convergence of the variable metric proximal point iterates. These results are applied to obtain conditions under which the Chen--Fukushima variable metric proximal point algorithm is super--linearly convergent wh...

Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at