## GLOBAL AND FINITE TERMINATION OF A TWO-PHASE AUGMENTED LAGRANGIAN FILTER METHOD FOR GENERAL QUADRATIC PROGRAMS (2007)

### Cached

### Download Links

Citations: | 1 - 0 self |

### BibTeX

@MISC{Friedlander07globaland,

author = {Michael P. Friedlander and Sven Leyffer},

title = {GLOBAL AND FINITE TERMINATION OF A TWO-PHASE AUGMENTED LAGRANGIAN FILTER METHOD FOR GENERAL QUADRATIC PROGRAMS },

year = {2007}

}

### OpenURL

### Abstract

We present a two-phase algorithm for solving large-scale quadratic programs (QPs). In the first phase, gradient-projection iterations approximately minimize a bound-constrained augmented Lagrangian function and provide an estimate of the optimal active set. In the second phase, an equality-constrained QP defined by the current active set is approximately minimized in order to generate a second-order search direction. A filter determines the required accuracy of the subproblem solutions and provides an acceptance criterion for the search directions. The resulting algorithm is globally and finitely convergent. The algorithm is suitable for large-scale problems with many degrees of freedom, and provides an alternative to interior-point methods when iterative methods must be used to solve the underlying linear systems. Numerical experiments on a subset of the CUTEr QP test problems demonstrate the effectiveness of the approach.

### Citations

1894 | Numerical Optimization - Nocedal, SJ - 2000 |

496 | Constrained Optimization and Lagrange Multiplier Method - Bertsekas - 1982 |

248 |
Interior-point polynomial methods in convex programming, ser
- Nesterov, Nemirovsky
- 1994
(Show Context)
Citation Context ... Science, U.S. Department of Energy, under Contract W-31-109-ENG-38 1s2 MICHAEL P. FRIEDLANDER AND SVEN LEYFFER vergence properties. For convex QPs, interior methods are convergent in polynomial time =-=[56]-=-. However, the key subproblems within these methods lead to linear systems (known as Karush-Kuhn-Tucker, or saddle-point, systems) that are inherently illconditioned [38, Theorem 4.2]. Implementations... |

179 | Numerical solution of saddle point problems
- Benzi, Golub, et al.
- 2005
(Show Context)
Citation Context ...ch we are able to solve the bound-constrained subproblem with a conjugate gradient method. We expect that preconditioners and iterative solvers that are specially designed for KKT systems (see, e.g., =-=[10]-=-) will dramatically improve this performance. We plan to explore this in future implementations. 6. Discussion and conclusions. We have presented a new active-set method for solving QPs that has the p... |

160 | Nonlinear Programming without a Penalty Function
- FLETCHER, LEYFFER
- 2002
(Show Context)
Citation Context ...n estimate of the optimal active set. With that active-set estimate, the second phase then solves an equality-constrained QP (it is this subproblem that gives rise to the KKT system). A filter method =-=[36]-=- is used to dynamically control the accuracy of the bound-constrained solves, thereby eliminating an arbitrary and sometimes troublesome sequence of parameters commonly used in augmented Lagrangian te... |

153 | Efficient management of parallelism in object oriented numerical software libraries - Balay, Gropp, et al. - 1997 |

144 | An Interior Point algorithm for nonconvex nonlinear programming
- Vanderbei, Shanno
- 1999
(Show Context)
Citation Context ...ause it is readily available within PETSc and because it simplifies our initial implementation. We compare our implementation with two general-purpose interior-point solvers: KNITRO [18, 62] and LOQO =-=[61]-=-. Although these methods are targeted to general nonlinear optimization problems, both solvers detect whether the problem is a QP and use appropriate algorithmic options. At this stage we are not inte... |

91 | Solving Unsymmetric Sparse Systems of Linear Equations with PARDISO
- Schenk, Gärtner
- 2004
(Show Context)
Citation Context ...it of 1000. Other linear solvers can easily be used within the PETSc framework, and specialized linear solvers and preconditioners for KKT systems are currently an active area of research (see, e.g., =-=[31,44,58]-=-). We have deliberately chosen the general-purpose solver GMRES because it is readily available within PETSc and because it simplifies our initial implementation. We compare our implementation with tw... |

87 |
LANCELOT: A Fortran package for Large-scale Nonlinear Optimization (Release
- Conn, Gould, et al.
- 1992
(Show Context)
Citation Context ... 7 . 8 k ← k + 1. Our algorithm may be interpreted as a second-order version of the classical augmented Lagrangian algorithm for nonlinear programming, as implemented in the software package LANCELOT =-=[22]-=-. We follow [40] and use the term bound-constrained Lagrangian (BCL) for these methods because they involve only bound constraints on each subproblem. Typically the original bound constraints are repe... |

76 | Interior methods for nonlinear optimization - Forsgren, Gill, et al. |

74 | Newton’s method for large boundconstrained optimization problems
- Lin, Moré
- 1999
(Show Context)
Citation Context ...rangian function (step 3 of Algorithm 2) and the solution of an EQP (step 5 of Algorithm 3). Our implementation uses the bound-constrained solver within TAO [8] (version 1.8.1) which is based on TRON =-=[52]-=- for step 3 of Algorithm 2. TAO’s flexible interface allows a user-defined termination criterion; we use this feature to implement the filter-based termination criterion defined in steps 11–12 of Algo... |

72 | Parallel Lagrange-Newton-Krylov-Schur methods for PDE-constrained optimization. Part II: The Lagrange-Newton solver, and its application to optimal control of steady viscous flows - Biros, Ghattas - 2000 |

56 | Preconditioners for Indefinite Systems Arising in Optimization - Gill, Murray, et al. - 1992 |

53 |
A globally convergent augmented Lagrangian algorithm for optimization with general constraints and simple bounds
- Toint
- 1991
(Show Context)
Citation Context ...cause they involve only bound constraints on each subproblem. Typically the original bound constraints are repeated verbatim in each subproblem and enforced at all iterations. See [13, Chapter 2] and =-=[23]-=- for an overview of BCL methods. BCL methods for convex QPs with general constraints have been recently considered by Dostál et al. [28–30] and by Delbos and Gilbert [24]. Active-set methods for solvi... |

49 | On the global convergence of an SLP-filter algorithm that takes EQP steps
- Chin, Fletcher
(Show Context)
Citation Context ...ming/equality-quadratic-programming methods, which have received much attention recently. For examples of such approaches, see Fletcher and Sainz de la Maza [35] and, more recently, Chin and Fletcher =-=[19]-=- and Byrd et al. [17]. A common approach of these methods is to solve a relatively inexpensive LP subproblem in order to estimate the optimal active set, and then solve an equality-constrained QP to o... |

49 | A Reflective Newton Method for Minimizing a Quadratic Function Subject to Bounds on Some of the Variables - Coleman, Li - 1996 |

44 | Preconditioning Indefinite Systems in Interior Point Methods for - Bergamaschi, Gondzio, et al. - 2004 |

41 | Augmented Lagrangian Methods under the Constant Positive Linear Dependence Constraint Qualification
- Andreani, Birgin, et al.
- 2004
(Show Context)
Citation Context ... must therefore be a solution of (2.12). In Theorem 7.1 (see section 7) we give an analogous convergence proof for a slightly modified version of Algorithm 2 that offers alternative to steps 5–6. See =-=[1]-=- for a related convergence analysis that relies on a set of different assumptions. The hypotheses of Theorem 3.4 can fail to hold if there are no convergent subsequences (i.e., Assumption 3.1 fails to... |

40 | An algorithm for nonlinear optimization using linear programming and equality constrained subproblems, Mathematical Programming 100
- Byrd, Gould, et al.
- 2004
(Show Context)
Citation Context ...ic-programming methods, which have received much attention recently. For examples of such approaches, see Fletcher and Sainz de la Maza [35] and, more recently, Chin and Fletcher [19] and Byrd et al. =-=[17]-=-. A common approach of these methods is to solve a relatively inexpensive LP subproblem in order to estimate the optimal active set, and then solve an equality-constrained QP to obtain a search direct... |

39 | KNITRO: An Integrated Package for Nonlinear Optimization”, Large-Scale Nonlinear Optimization
- Byrd, Nocedal, et al.
- 2006
(Show Context)
Citation Context ...e solver GMRES because it is readily available within PETSc and because it simplifies our initial implementation. We compare our implementation with two general-purpose interior-point solvers: KNITRO =-=[18, 62]-=- and LOQO [61]. Although these methods are targeted to general nonlinear optimization problems, both solvers detect whether the problem is a QP and use appropriate algorithmic options. At this stage w... |

39 |
CUTEr and SifDec: A constrained and unconstrained testing environment, revisited
- Toint
(Show Context)
Citation Context ...ite number of iterations, and an exact solve of (2.3) subsequently gives the solution of (GQP). 5. Numerical results. We implemented the QPFIL algorithm on a subset of medium-scale QPs from the CUTEr =-=[46]-=- test set, with the aim of demonstrating the global convergence and active-set identification properties of QPFIL. Our test problems are taken from the AMPL versions of the CUTEr test problems [60]; w... |

33 | A preconditioner for generalized saddle point problems - Benzi, Golub |

32 |
de la Maza, Nonlinear programming and nonsmooth optimization by successive linear programming
- Fletcher, Sainz
- 1989
(Show Context)
Citation Context ...is reminiscent of sequential linear-programming/equality-quadratic-programming methods, which have received much attention recently. For examples of such approaches, see Fletcher and Sainz de la Maza =-=[35]-=- and, more recently, Chin and Fletcher [19] and Byrd et al. [17]. A common approach of these methods is to solve a relatively inexpensive LP subproblem in order to estimate the optimal active set, and... |

32 | A globally convergent primal-dual interior-point filter method for nonlinear programming
- Ulbrich, Ulbrich, et al.
- 2004
(Show Context)
Citation Context ...2], and that ω(x, y) is the dual feasibility error. This definition of the filter is similar to the one used in [45]. The gradient of the Lagrangian has also been used in the filter by Ulbrich et al. =-=[59]-=-, together with a centrality measure, in the context of interior-point methods. 2.2. Active-set prediction and second-order steps. Let �x k be an approximate minimizer of the augmented Lagrangian L k ... |

31 | On the maximization of a concave quadratic function with box constraints
- Friedlander, Martínez
- 1994
(Show Context)
Citation Context ... projection to predict the optimal active set has been used in the context of bound-constrained QPs (i.e., with no general linear constraints) by Moré and Toraldo [55] and by Friedlander and Martínez =-=[39]-=-, among others. Bound-constrained QP solvers have also been considered by [6, 20, 25–27].s4 MICHAEL P. FRIEDLANDER AND SVEN LEYFFER Algorithm 1: Outline of QP Filter Method (QPFIL) initialization: k ←... |

31 |
Inertiacontrolling methods for general quadratic programming
- Gill, Murray, et al.
- 1991
(Show Context)
Citation Context ...he SQP framework because they can exploit increasingly good starting points in order to reduce the number of iterations required for convergence. Inertia-controlling active-set strategies (see, e.g., =-=[33,43]-=-) are robust in practice, but their overall efficiency is limited by the number of active-set changes that can be made at each iteration (typically, a single index changes at each iteration). The comb... |

28 | A constraint-stabilized time-stepping approach for rigid multibody dynamics with joints, contact and friction
- Anitescu, Hart
(Show Context)
Citation Context ...90C26, 90C52 1. Introduction. Quadratic programs (QPs) play a fundamental role in optimization. They are useful across a rich class of applications, such as the simulation of rigid multibody dynamics =-=[2, 50]-=-, optimal control [7, 32, 53], and financial-portfolio optimization [15,54]. They also arise as a sequence of subproblems within algorithms for solving more general nonlinear optimization problems. Of... |

23 | Optimization-based simulation of nonsmooth rigid multibody dynamics
- Anitescu
- 2006
(Show Context)
Citation Context ...90C26, 90C52 1. Introduction. Quadratic programs (QPs) play a fundamental role in optimization. They are useful across a rich class of applications, such as the simulation of rigid multibody dynamics =-=[2, 50]-=-, optimal control [7, 32, 53], and financial-portfolio optimization [15,54]. They also arise as a sequence of subproblems within algorithms for solving more general nonlinear optimization problems. Of... |

22 | Box constrained quadratic programming with controlled precision of auxiliary problems and applications - Dostál - 1996 |

21 | A globally convergent linearly constrained Lagrangian method for nonlinear optimization
- Friedlander, Saunders
(Show Context)
Citation Context .... Our algorithm may be interpreted as a second-order version of the classical augmented Lagrangian algorithm for nonlinear programming, as implemented in the software package LANCELOT [22]. We follow =-=[40]-=- and use the term bound-constrained Lagrangian (BCL) for these methods because they involve only bound constraints on each subproblem. Typically the original bound constraints are repeated verbatim in... |

18 |
A general quadratic programming algorithm
- Fletcher
- 1971
(Show Context)
Citation Context ...he SQP framework because they can exploit increasingly good starting points in order to reduce the number of iterations required for convergence. Inertia-controlling active-set strategies (see, e.g., =-=[33,43]-=-) are robust in practice, but their overall efficiency is limited by the number of active-set changes that can be made at each iteration (typically, a single index changes at each iteration). The comb... |

17 |
Testing a class of methods for solving minimization problems with simple bounds on the variables
- Toint
- 1988
(Show Context)
Citation Context ... With this goal in mind, we propose a new algorithm for solving QPs that is motivated by the computational effectiveness of gradient-projection methods (such as those described by [13, Chapter 2] and =-=[21]-=-) for bound-constrained QPs. A simplistic extension of gradient projection to general QPs would lead to a subproblem that is almost as difficult to solve as the original QP: each projection of the obj... |

14 |
MA57 - a code for the solution of sparse symmetric definite and indefinite systems
- Duff
(Show Context)
Citation Context ...it of 1000. Other linear solvers can easily be used within the PETSc framework, and specialized linear solvers and preconditioners for KKT systems are currently an active area of research (see, e.g., =-=[31,44,58]-=-). We have deliberately chosen the general-purpose solver GMRES because it is readily available within PETSc and because it simplifies our initial implementation. We compare our implementation with tw... |

11 | Inertia-controlling factorizations for optimization algorithms - Forsgren - 2002 |

11 |
User’s guide for SQOPT 5.3: A fortran package for large-scale linear and quadratic programming
- Gill, Murray, et al.
- 1997
(Show Context)
Citation Context ...raints have been recently considered by Dostál et al. [28–30] and by Delbos and Gilbert [24]. Active-set methods for solving large-scale nonconvex QPs include Galahad’s QPA [48], BQPD [34], and SQOPT =-=[42]-=-; see Gould and Toint [49] for a recent survey. 2. Augmented Lagrangian filter algorithm for QPs. Our algorithm differs from classical BCL method in three important ways: First, the main role of the a... |

10 |
TAO user manual (revision 1.8
- Benson, McInnes, et al.
(Show Context)
Citation Context ...tion of the bound-constrained augmented Lagrangian function (step 3 of Algorithm 2) and the solution of an EQP (step 5 of Algorithm 3). Our implementation uses the bound-constrained solver within TAO =-=[8]-=- (version 1.8.1) which is based on TRON [52] for step 3 of Algorithm 2. TAO’s flexible interface allows a user-defined termination criterion; we use this feature to implement the filter-based terminat... |

9 | SQP methods for large-scale nonlinear programming
- Gould, Toint
- 2000
(Show Context)
Citation Context ... problems. Of particular interest for us are sequential quadratic programming (SQP) methods, which have proved to be a reliable approach for general problems (for a recent survey, see Gould and Toint =-=[47]-=-). Our purpose is to develop a QP algorithm that may be used effectively within an SQP framework for solving large-scale nonlinear problems. Compared to interior-point methods for QPs, active-set meth... |

8 | Adaptive precision control in quadratic programming with simple bounds and/or equality constraints - Dostál, Friedlander, et al. - 1998 |

6 | An iterative working-set method for large-scale nonconvex quadratic programming
- Gould, Toint
(Show Context)
Citation Context ...nvex QPs with general constraints have been recently considered by Dostál et al. [28–30] and by Delbos and Gilbert [24]. Active-set methods for solving large-scale nonconvex QPs include Galahad’s QPA =-=[48]-=-, BQPD [34], and SQOPT [42]; see Gould and Toint [49] for a recent survey. 2. Augmented Lagrangian filter algorithm for QPs. Our algorithm differs from classical BCL method in three important ways: Fi... |

5 |
Global linear convergence of an augmented Lagrangian algorithm for solving convex quadratic optimization problems
- Gilbert
(Show Context)
Citation Context ...ions. See [13, Chapter 2] and [23] for an overview of BCL methods. BCL methods for convex QPs with general constraints have been recently considered by Dostál et al. [28–30] and by Delbos and Gilbert =-=[24]-=-. Active-set methods for solving large-scale nonconvex QPs include Galahad’s QPA [48], BQPD [34], and SQOPT [42]; see Gould and Toint [49] for a recent survey. 2. Augmented Lagrangian filter algorithm... |

5 | A multidimensional filter algorithm for nonlinear equations and nonlinear least-squares
- Gould, Leyffer, et al.
(Show Context)
Citation Context ...n that Ax − b is a steepest descent direction at x for the augmented Lagrangian [14, §2.2], and that ω(x, y) is the dual feasibility error. This definition of the filter is similar to the one used in =-=[45]-=-. The gradient of the Lagrangian has also been used in the filter by Ulbrich et al. [59], together with a centrality measure, in the context of interior-point methods. 2.2. Active-set prediction and s... |

5 |
Combining implicit restarts and partial reorthogonalization in Lanczos bidiagonalization, http://soi.stanford.edu/∼rmunk/PROPACK
- Larsen
(Show Context)
Citation Context ...est singular value of the current Ak which can be relatively expensive to compute. One possibility for estimating this value is to use a Lanczos bidiagonalization procedure, as implemented in PROPACK =-=[51]-=-. Ideally, we would compute the penalty value according to (2.4) or (2.5). However, for the size of problems of interest, this approach would be prohibitive in terms of computational effort. In our nu... |

4 |
Quadratic programming for large-scale portfolio optimization
- Best, Kale
- 2000
(Show Context)
Citation Context ...ole in optimization. They are useful across a rich class of applications, such as the simulation of rigid multibody dynamics [2, 50], optimal control [7, 32, 53], and financial-portfolio optimization =-=[15,54]-=-. They also arise as a sequence of subproblems within algorithms for solving more general nonlinear optimization problems. Of particular interest for us are sequential quadratic programming (SQP) meth... |

4 | An algebraic analysis of a block diagonal preconditioner for saddle point systems
- Golub, Greif, et al.
(Show Context)
Citation Context ...it of 1000. Other linear solvers can easily be used within the PETSc framework, and specialized linear solvers and preconditioners for KKT systems are currently an active area of research (see, e.g., =-=[31,44,58]-=-). We have deliberately chosen the general-purpose solver GMRES because it is readily available within PETSc and because it simplifies our initial implementation. We compare our implementation with tw... |

3 |
Computation of optimal controls by a method combining quasilinearization and quadratic programming
- Bashein, Enns
- 1972
(Show Context)
Citation Context ...ion. Quadratic programs (QPs) play a fundamental role in optimization. They are useful across a rich class of applications, such as the simulation of rigid multibody dynamics [2, 50], optimal control =-=[7, 32, 53]-=-, and financial-portfolio optimization [15,54]. They also arise as a sequence of subproblems within algorithms for solving more general nonlinear optimization problems. Of particular interest for us a... |

3 | Numerical analysis of leaving-face parameters in bound-constrained quadratic minimization - Diniz-Ehrhardt, Gomes-Ruggiero, et al. |

2 |
Stochastic and deterministic design and control via linear and quadratic programming
- Fegley, Blum, et al.
- 1971
(Show Context)
Citation Context ...ion. Quadratic programs (QPs) play a fundamental role in optimization. They are useful across a rich class of applications, such as the simulation of rigid multibody dynamics [2, 50], optimal control =-=[7, 32, 53]-=-, and financial-portfolio optimization [15,54]. They also arise as a sequence of subproblems within algorithms for solving more general nonlinear optimization problems. Of particular interest for us a... |

2 |
Benchmarks for nonlinear optimization
- Vanderbei
- 2002
(Show Context)
Citation Context ...Er [46] test set, with the aim of demonstrating the global convergence and active-set identification properties of QPFIL. Our test problems are taken from the AMPL versions of the CUTEr test problems =-=[60]-=-; we selected a subset of the test problems that had up to 20,000 variables or constraints. General inequality constraints were converted into equalities by introducing slack variables. The chosen tes... |

2 | Global linear convergence of an augmented lagrangian algorithm for solving convex quadratic optimization problems
- Delbos, Gilbert
(Show Context)
Citation Context ...ions. See [13, Chapter 2] and [23] for an overview of BCL methods. BCL methods for convex QPs with general constraints have been recently considered by Dostál et al. [28–30] and by Delbos and Gilbert =-=[24]-=-. Active-set methods for solving large-scale nonconvex QPs include Galahad’s QPA [47], BQPD [34], and SQOPT [41]; see Gould and Toint [48] for a recent survey. 2. Augmented Lagrangian filter algorithm... |

2 | A hard-constraint time-stepping approach for rigid multibody dynamics with joints, contact, and friction
- Hart, Anitescu
- 2003
(Show Context)
Citation Context ...90C20, 90C52 1. Introduction. Quadratic programs (QPs) play a fundamental role in optimization. They are useful across a rich class of applications, such as the simulation of rigid multibody dynamics =-=[2, 49]-=-, optimal control [7, 32, 52], and financial-portfolio optimization [15,53]. They also arise as a sequence of subproblems within algorithms for solving more general nonlinear optimization problems. Of... |