## A globally convergent Lagrangian barrier algorithm for optimization with general inequality constraints and simple bounds (1997)

### Cached

### Download Links

- [www.ams.org]
- [www.ams.org]
- [www.ams.org]
- [ftp.numerical.rl.ac.uk]
- DBLP

### Other Repositories/Bibliography

Venue: | MATH. OF COMPUTATION |

Citations: | 5 - 1 self |

### BibTeX

@ARTICLE{Conn97aglobally,

author = {A. R. Conn and Nick Gould and Ph. L. Toint},

title = {A globally convergent Lagrangian barrier algorithm for optimization with general inequality constraints and simple bounds},

journal = {MATH. OF COMPUTATION},

year = {1997},

volume = {66},

pages = {261--288}

}

### OpenURL

### Abstract

We consider the global and local convergence properties of a class of Lagrangian barrier methods for solving nonlinear programming problems. In such methods, simple bound constraints may be treated separately from more general constraints. The objective and general constraint functions are combined in a Lagrangian barrier function. A sequence of such functions are approximately minimized within the domain defined by the simple bounds. Global convergence of the sequence of generated iterates to a first-order stationary point for the original problem is established. Furthermore, possible numerical difficulties associated with barrier function methods are avoided as it is shown that a potentially troublesome penalty parameter is bounded away from zero. This paper is a companion to previous work of ours on augmented Lagrangian methods.

### Citations

1085 | Practical methods of optimization - Fletcher - 1991 |

652 | A new polynomial-time algorithm for linear programming
- Karmarkar
- 1984
(Show Context)
Citation Context ..., although their experience is only with simple bounds.sA GLOBALLY CONVERGENT LAGRANGIAN BARRIER ALGORITHM 263 Interest in the use of barrier functions was rekindled by the seminal paper of Karmarkar =-=[34]-=- on polynomial-time interior-point algorithms for linear programming and by the intimate connection between these methods and barrier function methods observed by Gill et al. [23]. The ill-conditionin... |

502 |
Constrained Optimization and Lagrange Multiplier Methods. Athena Scientific
- Bertsekas
- 1996
(Show Context)
Citation Context ...range multipliers at the solution, it is possible to avoid the need for µ to tend to zero and thus circumvent the conditioning problems inherent in the simple penalty function approach. See Bertsekas =-=[3]-=- and Conn et al. [11] for further details. It seems rather strange that such devices were not immediately applied to circumvent the conditioning difficulties associated with traditional barrier functi... |

435 |
Iterative Solution of Nonlinear Equations in Several Variables
- Ortega, Rheinboldt
- 2000
(Show Context)
Citation Context ...by giving a rate-of-convergence result for our algorithm in the spirit of Conn et al. [11, Theorem 5.5]. For a comprehensive discussion of convergence, the reader is referred to Ortega and Rheinboldt =-=[46]-=-.sA GLOBALLY CONVERGENT LAGRANGIAN BARRIER ALGORITHM 279 Theorem 5.6. Suppose that the iterates {xk} generated by Algorithm 3.1 satisfy AS6, that AS1 and AS3 hold and that λ∗ is the corresponding vect... |

315 |
Nonlinear Programming: Sequential Unconstrained Minimization Techniques
- Fiacco, McCormick
- 1968
(Show Context)
Citation Context ...or finding a local minimizer of (1.1) subject to a set of inequality constraints (1.2) was first introduced by Frisch [22]. The method was put on a sound theoretical framework by Fiacco and McCormick =-=[19]-=-, who also provide an interesting history of such techniques up until then. The basic idea is quite simple. A composite function, the barrier function, is constructed by combining the objective and co... |

178 |
A method for nonlinear constraints in minimization problems
- Powell
- 1969
(Show Context)
Citation Context ...there was almost immediately a way around the ill-conditioning difficulty, the development of augmented Lagrangian methods. These methods were introduced by Arrow and Solow [1], Hestenes [29], Powell =-=[50]-=- and Rockafellar [54]. The augmented Lagrangian function (corresponding to the quadratic penalty function (1.7)) for the above problem is f(x)+ 1 m� (ci(x)+si) 2µ 2 (1.8) , i=1s264 A. R. CONN, NICK GO... |

177 |
Multiplier and gradient methods
- Hestenes
- 1969
(Show Context)
Citation Context ...ime, though, there was almost immediately a way around the ill-conditioning difficulty, the development of augmented Lagrangian methods. These methods were introduced by Arrow and Solow [1], Hestenes =-=[29]-=-, Powell [50] and Rockafellar [54]. The augmented Lagrangian function (corresponding to the quadratic penalty function (1.7)) for the above problem is f(x)+ 1 m� (ci(x)+si) 2µ 2 (1.8) , i=1s264 A. R. ... |

135 | Nonlinear Programming: Analysis and Methods - Avriel - 2003 |

128 |
Augmented Lagrangians and applications of the proximal point algorithm in convex programming
- Rockafellar
- 1976
(Show Context)
Citation Context ...diately a way around the ill-conditioning difficulty, the development of augmented Lagrangian methods. These methods were introduced by Arrow and Solow [1], Hestenes [29], Powell [50] and Rockafellar =-=[54]-=-. The augmented Lagrangian function (corresponding to the quadratic penalty function (1.7)) for the above problem is f(x)+ 1 m� (ci(x)+si) 2µ 2 (1.8) , i=1s264 A. R. CONN, NICK GOULD, AND PH. L. TOINT... |

103 | Introduction to sensitivity and stability analysis in nonlinear programming - Fiacco - 1983 |

96 |
LANCELOT: A Fortran Package for LargeScale Nonlinear Optimization (Release A
- Toint
- 1991
(Show Context)
Citation Context ...-function method will be less sensitive to degeneracy for the general nonlinear problem than, for example, the implementation of a gradient-projection augmented Lagrangian method given by Conn et al. =-=[13]-=- in LANCELOT A. 3. There is numerical evidence that the method presented here is superior to standard barrier-function methods when applied to problems with simple bounds (see Conn et al. [17] and Nas... |

80 | Interior methods for constrained optimization - Wright - 1992 |

75 |
Projected gradient methods for linearly constrained problems
- CALAMAI, MORE
- 1987
(Show Context)
Citation Context ...equired at optimality for such a problem, (3.5) can be viewed as an inexact stopping rule for such iterative algorithms. We merely mention here that the projected gradient methods of Calamai and Moré =-=[8]-=-, Burke and Moré [6], Conn et al. [9], Conn et al. [10] and Burke et al. [7] and the interior-point method of Nash and Sofer [44] are all appropriate, but that methods which take special account of th... |

72 |
Computational experience with a primal-dual interior point method for linear programming. Linear Algebra and Its Applications
- Lustig, Marsten, et al.
- 1991
(Show Context)
Citation Context ...ate that barrier functions provide a way of determining which constraints are active at the solution in a significantly more efficient fashion than active set methods (see, for example, Lustig et al. =-=[37]-=-). More recent experience with the same class of problems using shifted barrier methods has been equally encouraging (see Jensen et al. [32]). The theory presented below for the Lagrangian barrier fun... |

67 | On the projected Newton barrier methods for linear programming and an equivalence to Karmarkar’s projective method, Mathematical Programming 36
- Gill, Murray, et al.
- 1986
(Show Context)
Citation Context ...l paper of Karmarkar [34] on polynomial-time interior-point algorithms for linear programming and by the intimate connection between these methods and barrier function methods observed by Gill et al. =-=[23]-=-. The ill-conditioning problems described above do not occur for primal and dual nondegenerate linear programs as the solutions to such problems occur at vertices of the constraint boundary. Furthermo... |

62 |
Global convergence of a class of trust region algorithms for optimization with simple bounds
- Toint
- 1988
(Show Context)
Citation Context ...lem, (3.5) can be viewed as an inexact stopping rule for such iterative algorithms. We merely mention here that the projected gradient methods of Calamai and Moré [8], Burke and Moré [6], Conn et al. =-=[9]-=-, Conn et al. [10] and Burke et al. [7] and the interior-point method of Nash and Sofer [44] are all appropriate, but that methods which take special account of the nature of (1.4) may yet be preferre... |

54 |
On the identification of active constraints
- Burke, Moré
- 1988
(Show Context)
Citation Context ...y for such a problem, (3.5) can be viewed as an inexact stopping rule for such iterative algorithms. We merely mention here that the projected gradient methods of Calamai and Moré [8], Burke and Moré =-=[6]-=-, Conn et al. [9], Conn et al. [10] and Burke et al. [7] and the interior-point method of Nash and Sofer [44] are all appropriate, but that methods which take special account of the nature of (1.4) ma... |

53 |
A globally convergent augmented Lagrangian algorithm for optimization with general constraints and simple bounds
- Toint
- 1991
(Show Context)
Citation Context ... the solution, it is possible to avoid the need for µ to tend to zero and thus circumvent the conditioning problems inherent in the simple penalty function approach. See Bertsekas [3] and Conn et al. =-=[11]-=- for further details. It seems rather strange that such devices were not immediately applied to circumvent the conditioning difficulties associated with traditional barrier function methods, but this ... |

49 |
The logarithmic potential method of convex programming
- Frisch
- 1955
(Show Context)
Citation Context ...s are always satisfied. 1.1. Background. The logarithmic-barrier function method for finding a local minimizer of (1.1) subject to a set of inequality constraints (1.2) was first introduced by Frisch =-=[22]-=-. The method was put on a sound theoretical framework by Fiacco and McCormick [19], who also provide an interesting history of such techniques up until then. The basic idea is quite simple. A composit... |

41 |
Modified barrier functions: Theory and methods
- Polyak
- 1992
(Show Context)
Citation Context ...rithm considered in this paper are the shifted-barrier method analyzed for linear programs by Gill et al. [24] and the class of modified barrier methods proposed by Polyak [48] and analyzed in Polyak =-=[47]-=-. Gill et al. consider the shifted barrier function (1.10) f(x) − m� wi log(ci(x)+si), i=1 where the wi are termed weights and the si called shifts. A sequence of shifted barrier functions are minimiz... |

30 |
Interior-Point Methods for Convex Programming
- Jarre
- 1990
(Show Context)
Citation Context ...odern interior-point approaches that are based upon extensions of methods for linear and quadratic programming, our approach makes no convexity or self-concordancy assumptions (see Jarre and Saunders =-=[30]-=- and Nesterov and Nemirovsky [45]). 1 It is also rather curious to note the strong similarity between (1.8) and the first two terms of a Taylor’s expansion of (1.14) for small ci(x)/si. Thus, one migh... |

29 |
On the accurate determination of search directions for simple differentiable penalty functions
- Gould
- 1986
(Show Context)
Citation Context ...uently, the initial enthusiasm for barrier function methods declined. Methods which alleviate these difficulties have been proposed (see, e.g., Murray [40], Wright [55], Murray and Wright [42], Gould =-=[25]-=-, and McCormick [38]) that are immediately applicable to smaller dense problems. Nash and Sofer [44] have recently discussed an approach that is applicable to large-scale, nonlinear problems, although... |

25 |
Convergence properties of trust region methods for linear and convex constraints
- Burke, Moré, et al.
- 1990
(Show Context)
Citation Context ...stopping rule for such iterative algorithms. We merely mention here that the projected gradient methods of Calamai and Moré [8], Burke and Moré [6], Conn et al. [9], Conn et al. [10] and Burke et al. =-=[7]-=- and the interior-point method of Nash and Sofer [44] are all appropriate, but that methods which take special account of the nature of (1.4) may yet be preferred.sA GLOBALLY CONVERGENT LAGRANGIAN BAR... |

22 |
Self--concordant functions and polynomial--time methods in convex programming
- Nesterov, Nemirovsky
- 1989
(Show Context)
Citation Context ...hat are based upon extensions of methods for linear and quadratic programming, our approach makes no convexity or self-concordancy assumptions (see Jarre and Saunders [30] and Nesterov and Nemirovsky =-=[45]-=-). 1 It is also rather curious to note the strong similarity between (1.8) and the first two terms of a Taylor’s expansion of (1.14) for small ci(x)/si. Thus, one might assume that for small ci(x)/si,... |

21 |
Analytical expressions for eigenvalues and eigenvectors of the Hessian matrices of barrier and penalty functions
- Murray
- 1971
(Show Context)
Citation Context ...t survey by Wright [56]. It was originally envisaged that each of the sequence of barrier functions be minimized using standard methods for unconstrained minimization. However Lootsma [36] and Murray =-=[40]-=- painted a less optimistic picture by showing that, under most circumstances, the spectral condition number of the Hessian matrix of the barrier function increases without bound as µ shrinks. This has... |

20 |
A barrier method for largescale constrained optimization
- Nash, Sofer
- 1993
(Show Context)
Citation Context ...difficulties have been proposed (see, e.g., Murray [40], Wright [55], Murray and Wright [42], Gould [25], and McCormick [38]) that are immediately applicable to smaller dense problems. Nash and Sofer =-=[44]-=- have recently discussed an approach that is applicable to large-scale, nonlinear problems, although their experience is only with simple bounds.sA GLOBALLY CONVERGENT LAGRANGIAN BARRIER ALGORITHM 263... |

17 |
Testing a class of methods for solving minimization problems with simple bounds on the variables
- Toint
- 1988
(Show Context)
Citation Context ... viewed as an inexact stopping rule for such iterative algorithms. We merely mention here that the projected gradient methods of Calamai and Moré [8], Burke and Moré [6], Conn et al. [9], Conn et al. =-=[10]-=- and Burke et al. [7] and the interior-point method of Nash and Sofer [44] are all appropriate, but that methods which take special account of the nature of (1.4) may yet be preferred.sA GLOBALLY CONV... |

14 |
On the number of inner iterations per outer iteration of a globally convergent algorithm for optimization with general nonlinear equality constraints and simple bounds
- Toint
- 1992
(Show Context)
Citation Context ... as input to the next. This is not specific to the algorithm discussed in this paper, but also applies, for instance, to augmented Lagrangian methods, as discussed by Conn et al. [11] and Conn et al. =-=[14]-=-. We also note that a result similar to that of the latter reference can also be shown, under certain additional assumptions, for the algorithm considered here, namely that (3.8) is eventually satisfi... |

13 |
Hessian matrices of penalty functions for solving constrainedoptimization problems
- Lootsma
- 1969
(Show Context)
Citation Context ...n, see the recent survey by Wright [56]. It was originally envisaged that each of the sequence of barrier functions be minimized using standard methods for unconstrained minimization. However Lootsma =-=[36]-=- and Murray [40] painted a less optimistic picture by showing that, under most circumstances, the spectral condition number of the Hessian matrix of the barrier function increases without bound as µ s... |

12 |
Numerical Methods for Nonlinearly Constrained Optimization
- Wright
- 1976
(Show Context)
Citation Context ...unter numerical difficulties. Consequently, the initial enthusiasm for barrier function methods declined. Methods which alleviate these difficulties have been proposed (see, e.g., Murray [40], Wright =-=[55]-=-, Murray and Wright [42], Gould [25], and McCormick [38]) that are immediately applicable to smaller dense problems. Nash and Sofer [44] have recently discussed an approach that is applicable to large... |

11 |
Gradient methods for constrained maxima with weakened assumptions
- Arrow, Solow
- 1958
(Show Context)
Citation Context ... (1.7). This time, though, there was almost immediately a way around the ill-conditioning difficulty, the development of augmented Lagrangian methods. These methods were introduced by Arrow and Solow =-=[1]-=-, Hestenes [29], Powell [50] and Rockafellar [54]. The augmented Lagrangian function (corresponding to the quadratic penalty function (1.7)) for the above problem is f(x)+ 1 m� (ci(x)+si) 2µ 2 (1.8) ,... |

10 | Interior Point Methods for Mathematical Programming: a Bibliography, Diskussionsbeitrag Nr
- Kranich
- 1991
(Show Context)
Citation Context ...al methods may be used to solve the problems (Murray [41]). Moreover, and most significantly, these methods have turned out to be most effective in practice (see the excellent bibliography of Kranich =-=[35]-=-). However, it is quite surprising how the lessons of the early 1970s seem to have been forgotten in the rush to extend interior-point methods to solving general constrained optimization problems. The... |

10 |
The superlinear convergence of a nonlinear primal-dual algorithm
- McCormick
- 1991
(Show Context)
Citation Context ...enthusiasm for barrier function methods declined. Methods which alleviate these difficulties have been proposed (see, e.g., Murray [40], Wright [55], Murray and Wright [42], Gould [25], and McCormick =-=[38]-=-) that are immediately applicable to smaller dense problems. Nash and Sofer [44] have recently discussed an approach that is applicable to large-scale, nonlinear problems, although their experience is... |

9 |
Preliminary Computational Experience with Modi ed LogBarrier Functions for Large-Scale Nonlinear programming ", Large Scale Optimization State of the
- Breitfeld, Shanno
- 1994
(Show Context)
Citation Context ...iers are adjusted will converge to a solution of the original problem. This has the desirable effect of limiting the size of the condition number of the Hessian matrix of (1.11). Breitfeld and Shanno =-=[4]-=- point out that this additional flexibility allows the incorporation of equality constraints via two (shifted) inequalities. Finally, Freund [21], Jensen et al. [32] and Powell [51] have analyzed and ... |

9 |
Computational experience with penalty/barrier methods for nonlinear programming
- Breitfeld, Shanno
- 1996
(Show Context)
Citation Context ...simple bounds (see Conn et al. [17] and Nash et al. [43]), and preliminary evidence suggests that the same is true for more general constraints (see Breitfeld and Shanno [4], and Breitfeld and Shanno =-=[5]-=-). 4. Unlike many modern interior-point approaches that are based upon extensions of methods for linear and quadratic programming, our approach makes no convexity or self-concordancy assumptions (see ... |

9 | A globally convergent Lagrangian barrier algorithm for optimization with general inequality constraints and simple bounds - Toint - 1992 |

9 | Degeneracy in interior point methods for linear programming: a survey
- GÜLER, HERTOG, et al.
- 1993
(Show Context)
Citation Context ...ior-point methods for linear programming also suggests that these methods are less sensitive to degeneracy than methods that attempt to identify activities explicitly (see, for instance, Güler et al. =-=[27]-=-, and Rendl et al. [53]). Based on this experience, there is hope that a shifted barrier-function method will be less sensitive to degeneracy for the general nonlinear problem than, for example, the i... |

8 |
A note on using alternative second-order models for the subproblems arising in barrier function methods for minimization
- Toint
- 1994
(Show Context)
Citation Context ... et al. [13] in LANCELOT A. 3. There is numerical evidence that the method presented here is superior to standard barrier-function methods when applied to problems with simple bounds (see Conn et al. =-=[17]-=- and Nash et al. [43]), and preliminary evidence suggests that the same is true for more general constraints (see Breitfeld and Shanno [4], and Breitfeld and Shanno [5]). 4. Unlike many modern interio... |

8 |
A catalogue of subroutines (release 11
- Library
- 1993
(Show Context)
Citation Context ...ently than the more sophisticated implementation of an augmented Lagrangian algorithm in our Fortran package, LANCELOT A. Moreover, considerable success was reported in the Harwell Subroutine Library =-=[28]-=- code VE14, a Lagrangian Barrier algorithm for bound-constrained quadratic programming problems. 9. Acknowledgements The authors are grateful to Roman Polyak for the stimulating conversations concerni... |

8 |
Projected Lagrangian Methods Based on the Trajectories of Penalty and Barrier Functions
- Murray, Wright
- 1978
(Show Context)
Citation Context ...ties. Consequently, the initial enthusiasm for barrier function methods declined. Methods which alleviate these difficulties have been proposed (see, e.g., Murray [40], Wright [55], Murray and Wright =-=[42]-=-, Gould [25], and McCormick [38]) that are immediately applicable to smaller dense problems. Nash and Sofer [44] have recently discussed an approach that is applicable to large-scale, nonlinear proble... |

8 |
Barrier methods for large-scale quadratic programming
- Ponceleón
- 1991
(Show Context)
Citation Context ...us to be the observation that, although the ill-conditioning difficulties are present in most nonlinear programs, the effects may be benign provided sufficient care is taken. In particular, Ponceleón =-=[49]-=- has shown that if the only constraints that are handled by logarithmic terms are simple bounds, the ill-conditioning manifests itself solely on the diagonal of the Hessian matrix of the barrier funct... |

8 | Algorithmic Methods in Optimal Control - Gruver, Sachs - 1981 |

7 |
Shifted barrier methods for linear programming
- Gill, Murray, et al.
- 1988
(Show Context)
Citation Context ...e Lagrange multiplier estimates carefully. The methods which are closest in spirit to the algorithm considered in this paper are the shifted-barrier method analyzed for linear programs by Gill et al. =-=[24]-=- and the class of modified barrier methods proposed by Polyak [48] and analyzed in Polyak [47]. Gill et al. consider the shifted barrier function (1.10) f(x) − m� wi log(ci(x)+si), i=1 where the wi ar... |

6 |
Theoretical efficiency of a shifted-barrier-function algorithm for linear programming
- Freund
- 1991
(Show Context)
Citation Context ...r of the Hessian matrix of (1.11). Breitfeld and Shanno [4] point out that this additional flexibility allows the incorporation of equality constraints via two (shifted) inequalities. Finally, Freund =-=[21]-=-, Jensen et al. [32] and Powell [51] have analyzed and implemented shifted and modified barrier function methods for linear programming. Jensen et al. [31] extend this work to convex problems.sA GLOBA... |

6 |
Convergence bounds for nonlinear programming algorithms
- Mifflin
- 1975
(Show Context)
Citation Context ...lementary slackness assumption, the error in solving (1.5), that is, the difference between the minimizer of (1.5) and the solution to the original problem, is of order µ as µ tends to zero. (Mifflin =-=[39]-=- shows an order √ µ error in the absence of the complementary slackness assumption and a weakening of the assumption that (1.5) be solved exactly.) For further discussion, see the recent survey by Wri... |

6 |
A numerical comparison of barrier and modified barrier methods for large-scale constrained optimization
- Nash, Polyak, et al.
- 1994
(Show Context)
Citation Context ...LOT A. 3. There is numerical evidence that the method presented here is superior to standard barrier-function methods when applied to problems with simple bounds (see Conn et al. [17] and Nash et al. =-=[43]-=-), and preliminary evidence suggests that the same is true for more general constraints (see Breitfeld and Shanno [4], and Breitfeld and Shanno [5]). 4. Unlike many modern interior-point approaches th... |

5 |
A Modified Barrier Function Method with Improved Rate of Convergence for Degenerate Problems
- Jittorntrum, Osborne
- 1980
(Show Context)
Citation Context ...itioning difficulties associated with traditional barrier function methods, but this appears to be the case. To our knowledge, the first move in this direction was the work by Jittorntrum and Osborne =-=[33]-=- in which the authors consider a sequential minimization of the modified barrier function m� (1.9) f(x) − µ λi log(ci(x)) i=1 for appropriate Lagrange multiplier estimates λi. They show that it is pos... |

5 |
Some Convergence Properties of the Modified Log Barrier Method for Linear Programming
- Powell
- 1995
(Show Context)
Citation Context ...estimates ¯λk for inactive constraints in Theorem 4.2, (iv), is a consequence of our shift strategy. Only a linear convergence rate is proved by Polyak [47] for his modified barrier method and Powell =-=[52]-=- indicates that this can be a disadvantage when inactive constraints are relatively small at a limit point. We note that the modified barrier method of Jittorntrum and Osborne [33] also exhibits the s... |

4 |
On the convergence of a modified barrier method for convex programming
- Jensen, Polyak
- 1994
(Show Context)
Citation Context ... two (shifted) inequalities. Finally, Freund [21], Jensen et al. [32] and Powell [51] have analyzed and implemented shifted and modified barrier function methods for linear programming. Jensen et al. =-=[31]-=- extend this work to convex problems.sA GLOBALLY CONVERGENT LAGRANGIAN BARRIER ALGORITHM 265 1.2. Motivation. In this paper, we consider the Lagrangian barrier function (1.4). This function is of the ... |

3 |
A note on exploiting structure when using slack variables
- Toint
- 1994
(Show Context)
Citation Context ...t, is the need to introduce slack variables (see, e.g., Fletcher [20, p. 146]) to convert the inequalities to the form (1.6). Although any slack variables might be treated specially (see, Conn et al. =-=[16]-=-), there is still likely to be an overhead incurred from the increase in the number of unknowns. Thus, it would seem to be preferable to avoid slack variables if at all possible. The combination of re... |