Lagrange multipliers used to be viewed as auxiliary variables introduced in a problem of constrained minimization in order to write first-order optimality conditions formally as a system of equations. Modern applications, with their emphasis on numerical methods and more complicated side conditions than equations, have demanded deeper understanding of the concept and how it fits into a larger theoretical picture. A major line of research has been the nonsmooth geometry of one-sided tangent and normal vectors to the set of points satisfying the given constraints. Another has been the game-theoretic role of multiplier vectors as solutions to a dual problem. Interpretations as generalized derivatives of the optimal value with respect to problem parameters have also been explored. Lagrange multipliers are now being seen as arising from a general rule for the subdifferentiation of a nonsmooth objective function which allows black-and-white constraints to be replaced by penalty expressions. This paper traces such themes in the current theory of Lagrange multipliers, providing along the way a freestanding exposition of basic nonsmooth analysis as motivated by and applied to this subject.

Abstract. Many large-scale problems in dynamic and stochastic optimization can be modeled with extended linear-quadratic programming, which admits penalty terms and treats them through duality. In general the objective functions in such problems are only piecewise smooth and must be minimized or maximized relative to polyhedral sets of high dimensionality. This paper proposes a new class of numerical methods for “fully quadratic ” problems within this framework, which exhibit second-order nonsmoothness. These methods, combining the idea of finite-envelope representation with that of modified gradient projection, work with local structure in the primal and dual problems simultaneously, feeding information back and forth to trigger advantageous restarts. Versions resembling steepest descent methods and conjugate gradient methods are presented. When a positive threshold of ε-optimality is specified, both methods converge in a finite number of iterations. With threshold 0, it is shown under mild assumptions that the steepest descent version converges linearly, while the conjugate gradient version still has a finite termination property. The algorithms are designed to exploit features of primal and dual decomposability of the Lagrangian, which are typically available in a large-scale setting, and they are open to considerable parallelization. Key words. Extended linear-quadratic programming, large-scale numerical optimization, finite-envelope representation, gradient projection, primal-dual methods, steepest descent methods, conjugate gradient methods. AMS(MOS) subject classifications. 65K05, 65K10, 90C20 1. Introduction. A

this paper is to show that, under very mild assumptions, the Mararos effect cannot occur in the unconstrained minimization of SC

. Quadratic stochastic programs (QSP) with recourse can be formulated as nonlinear convex programming problems. By attaching a Lagrange multiplier vector to the nonlinear convex program, a QSP is written as a system of nonsmooth equations. A Newton-like method for solving the QSP is proposed and global convergence and local superlinear convergence of the method are established. The current method is more general than previous methods which were developed for box-diagonal and fully quadratic QSP. Numerical experiments are given to demonstrate the efficiency of the algorithm, and to compare the use of Monte-Carlo rules and lattice rules for multiple integration in the algorithm. Keywords: Newton's method, quadratic stochastic programs, nonsmooth equations. Short title: Newton's method for stochastic programs 1 This work is supported by the Australian Research Council. 1. Introduction Let P 2 R n\Thetan be symmetric positive semi-definite and H 2 R m\Thetam be symmetric positive...

Abstract. Optimization problems in discrete time can be modeled more flexibly by extended linearquadratic programming than by traditional linear or quadratic programming, because penalties and other expressions that may substitute for constraints can readily be incorporated and dualized. At the same time, dynamics can be written with state vectors as in dynamic programming and optimal control. This suggests new primal-dual approaches to solving multistage problems. The special setting for such numerical methods is described. New results are presented on the calculation of gradients of the primal and dual objective functions and on the convergence effects of strict quadratic regularization.

We consider the sequential quadratic programming algorithm (SQP) applied to equalityconstrained optimization problems, where the problem data is differentiable with Lipschitzcontinuous first derivatives. For this setting, Dennis-Moré type analysis of primal superlinear convergence is presented. Our main motivation is a special modification of SQP tailored to the structure of the lifted reformulation of mathematical programs with complementarity constraints (MPCC). For this problem, we propose a special positive definite modification of the matrices in the generalized Hessian, which is suitable for globalization of SQP based on the penalty function, and at the same time can be expected to satisfy our general Dennis-Moré type conditions, thus preserving local superlinear convergence. (Standard quasi-Newton updates in the SQP framework require twice differentiability of the problem data at the solution for superlinear convergence.) Preliminary numerical results comparing a number of quasi-Newton versions of semismooth SQP applied to MPCC are also reported. Key words: sequential quadratic programming, semismoothness, B-differential, BD-regularity, semismooth Newton method, second-order sufficiency, mathematical programs with complementarity constraints.

We consider the sequential quadratic programming algorithm (SQP) applied to equalityconstrained optimization problems, where the problem data is differentiable with Lipschitzcontinuous first derivatives. For this setting, Dennis-Moré type analysis of primal superlinear convergence is presented. Our main motivation is a special modification of SQP tailored to the structure of the lifted reformulation of mathematical programs with complementarity constraints (MPCC). For this problem, we propose a special positive definite modification of the matrices in the generalized Hessian, which is suitable for globalization of SQP based on the penalty function, and at the same time can be expected to satisfy our general Dennis-Moré type conditions, thus preserving local superlinear convergence. (Standard quasi-Newton updates in the SQP framework require twice differentiability of the problem data at the solution for superlinear convergence.) Preliminary numerical results comparing a number of quasi-Newton versions of semismooth SQP applied to MPCC are also reported. Keywords: sequential quadratic programming; semismoothness; B-differential; BD-regularity; semismooth Newton method; second-order sufficiency; mathematical programs with complementarity constraints AMS Subject Classification: 90C30; 90C33; 90C55; 65K05 1.

While generalized equations with differentiable single-valued base mappings and the associated Josephy–Newton method have been studied extensively, the setting with semismooth base mapping had not been previously considered (apart from the two special cases of usual nonlinear equations and of Karush-Kuhn-Tucker optimality systems). We introduce for the general semismooth case appropriate notions of solution regularity and prove local convergence of the corresponding Josephy–Newton method. As an application, we immediately recover the known primal-dual local convergence properties of semismooth SQP, but also obtain some new results that complete the analysis of the SQP primal rate of convergence, including its quasi-Newton variant. Key words: generalized equation, B-differential, generalized Jacobian, BD-regularity, CD-regularity,

We prove a new local upper Lipschitz stability result and the associated local error bound for solutions of parametric Karush–Kuhn–Tucker systems corresponding to variational problems with Lipschitzian base mappings and constraints possessing Lipschitzian derivatives, and without any constraint qualifications. This property is equivalent to the appropriately extended to this nonsmooth setting notion of noncriticality of the Lagrange multiplier associated to the primal solution, which is weaker than second-order sufficiency. All this extends several results previously known only for optimization problems with twice differentiable data, or assuming some constraint qualifications. In addition, our results are obtained in the more general variational setting.