## Interior methods for nonlinear optimization (2002)

Venue: | SIAM Review |

Citations: | 90 - 4 self |

### BibTeX

@ARTICLE{Forsgren02interiormethods,

author = {Anders Forsgren and Philip E. Gill and Margaret H. Wright},

title = {Interior methods for nonlinear optimization},

journal = {SIAM Review},

year = {2002},

volume = {44},

pages = {525--597}

}

### Years of Citing Articles

### OpenURL

### Abstract

Abstract. Interior methods are an omnipresent, conspicuous feature of the constrained optimization landscape today, but it was not always so. Primarily in the form of barrier methods, interior-point techniques were popular during the 1960s for solving nonlinearly constrained problems. However, their use for linear programming was not even contemplated because of the total dominance of the simplex method. Vague but continuing anxiety about barrier methods eventually led to their abandonment in favor of newly emerging, apparently more efficient alternatives such as augmented Lagrangian and sequential quadratic programming methods. By the early 1980s, barrier methods were almost without exception regarded as a closed chapter in the history of optimization. This picture changed dramatically with Karmarkar’s widely publicized announcement in 1984 of a fast polynomial-time interior method for linear programming; in 1985, a formal connection was established between his method and classical barrier methods. Since then, interior methods have advanced so far, so fast, that their influence has transformed both the theory and practice of constrained optimization. This article provides a condensed, selective look at classical material and recent research about interior methods for nonlinearly constrained optimization.

### Citations

3839 | Convex Analysis
- Rockafellar
- 1970
(Show Context)
Citation Context ... KKT conditions (2.2a)–(2.2d) are not enough to ensure optimality unless problem (2.1) is a convex program (Definition A.8). (For discussions of convex programming and convex analysis,see,for example,=-=[12,89]-=-.) Consequently,it is common to consider second-order optimality conditions as well. In this regard,the Hessian of the Lagrangian L(x, λ) (2.3) with respect to x will be crucial: (2.11) H(x, λ) △ = ∇ ... |

2248 |
Numerical optimization
- Nocedal, Wright
- 2000
(Show Context)
Citation Context ...t seemed to be more efficient in practice without being plagued by unavoidable ill-conditioning. In particular,augmented Lagrangian and sequential quadratic programming (SQP) methods (see,for example,=-=[6,34,52,77,80]-=-) are based directly on the optimality conditions for constrained optimization. Barrier methods appeared distinctly unappealing by comparison,and almost all researchers in mainstream optimization lost... |

1571 |
Theory of linear and integer programming
- Schrijver
- 1986
(Show Context)
Citation Context ...haved n-variable linear programs with m inequality constraints is that a vertex minimizer must exist,i.e.,a point where n constraints with linearly independent gradients hold with equality. (See,e.g.,=-=[20,92]-=- for details about linear programming.) The simplex method,invented by Dantzig in 1947,is an iterative procedure that solves linear programs by exploiting this property. A simplex iteration moves from... |

1569 |
Practical Optimization
- Gill, Murray, et al.
(Show Context)
Citation Context ...t seemed to be more efficient in practice without being plagued by unavoidable ill-conditioning. In particular,augmented Lagrangian and sequential quadratic programming (SQP) methods (see,for example,=-=[6,34,52,77,80]-=-) are based directly on the optimality conditions for constrained optimization. Barrier methods appeared distinctly unappealing by comparison,and almost all researchers in mainstream optimization lost... |

1082 |
Numerical Methods For Unconstrained Optimization and Nonlinear Equations
- Dennis, Schnabel
- 1996
(Show Context)
Citation Context ...tion on the size of �p�,where various norms may appear in the trust-region constraint (which,as its name suggests,specifies the local region in which the Taylor-series model can be trusted). See,e.g.,=-=[28,34,52,77,80]-=- for material on line-search methods,and [25,28,34,80] for discussion of trust-region methods. The presentation in section 3.3 focused on unconstrained minimizers of the log barrier function; hence we... |

851 | Semidefinite programming
- Vandenberghe, Boyd
- 1996
(Show Context)
Citation Context ... subject to semidefiniteness constraints—is arguably the most notable of these problems to receive widespread attention as a direct result of the development of interior methods (see,e.g.,the surveys =-=[65,99,101]-=-). The evident similarity of interior methods to longstanding continuation approaches (see,e.g.,[1,2]) has been recognized since the early days of modern interior methods (see,e.g.,[71]),but numerous ... |

695 | A new polynomial-time algorithm for linear programming
- Karmarkar
- 1984
(Show Context)
Citation Context ...implex method,and the search continued for a polynomial-time linear programming method that was genuinely fast in running time. The start of the interior-point revolution was Karmarkar’s announcement =-=[63]-=- in 1984 of a polynomial-time linear programming method that was 50 times faster than the simplex method. Amid the frenzy of interest in Karmarkar’s method,it was shown in 1985 [51] that there was a f... |

595 | Rheinboldt, Iterative Solution of Nonlinear Equations - Ortega, C - 1970 |

512 | Primal-Dual Interior-Point Methods
- Wright
- 1997
(Show Context)
Citation Context ...vergence.) Moreover,the inner iterations can be terminated so that the combined sequence of inner iterates ultimately converges to x ∗ Q-superlinearly if µ is reduced at an appropriate rate; see,e.g.,=-=[16,59,115,124,125]-=-. Beyond the work associated with evaluating the objective function,constraint functions,and their first and second derivatives,the cost of a primal-dual iteration is dominated by the cost of solving ... |

386 |
Interior-point polynomial algorithms in convex programming
- Nesterov, Nemirovskii
- 1994
(Show Context)
Citation Context ...l their dependence on properties of these paths,which provide the foundation for all complexity analyses of interior-point algorithms for linear, quadratic,and convex programming. The monumental work =-=[79]-=- of Nesterov and Nemirovskii proposed new families of barrier methods and extended polynomial-time complexity results to new convex optimization problems. Semidefinite programming—minimization of a co... |

194 |
Practical Methods of Optimization, 2nd ed
- Fletcher
- 1987
(Show Context)
Citation Context ...f an optimal multiplier is known, x ∗ can be found from just one unconstrained minimization. (The motivation for modified barrier methods is similar to that for augmented Lagrangian methods; see,e.g.,=-=[34,52,77,80]-=-.)s574 ANDERS FORSGREN, PHILIP E. GILL, AND MARGARET H. WRIGHT In practice,neither the optimal multiplier vector nor an upper bound on µ is known in advance. As a result,a sequence of problems must be... |

191 |
A method for nonlinear constraints in minimization problems
- Powell
- 1969
(Show Context)
Citation Context ...ack), ci(x)+si ≥ 0, si = 0 (shift). Both of these ideas have a long history in optimization. Slack variables pervade linear programming,and the idea of shifted constraints underlies the motivation in =-=[86]-=- of augmented Lagrangian methods for nonlinearly constrained optimization. We consider only shifts of the form (6.26b); for more general discussions of shifted constraints in interior methods,see,e.g.... |

187 | 2002), “Nonlinear programming without a penalty function
- Fletcher, Leyffer
(Show Context)
Citation Context ...cture,where the inertia of these systems can be determined without recourse to a special factorization method. A quite different procedure for forcing convergence is to use a filter,first proposed in =-=[36]-=- in the context of SQP methods. Constrained optimization is a “two-objective” problem in which we would like to minimize (i) the objective function and (ii) the norm of the constraint violations. Sinc... |

178 |
How good is the simplex algorithm
- Klee, Minty
- 1972
(Show Context)
Citation Context ...aints can hold with equality. In fact,with standard pivoting rules specifying the constraint to be dropped and added at each iteration,the simplex method can visit every vertex of the feasible region =-=[64]-=-; thus its worst-case complexity is exponential in the problem dimension. As a result,there was great interest in finding a polynomial-time linear programming algorithm. 1 The first success in this di... |

175 | An interior-point algorithm for nonconvex nonlinear programming
- Vanderbei, Shanno
- 1999
(Show Context)
Citation Context ...s in the classical primal barrier method,primal-dual methods based on properties of x(µ) are increasingly popular for solving general nonlinear programming problems; see,for example,the recent papers =-=[14,24,31,39,46,93,97,103]-=-. As in primal-dual methods for linear programming,the original (primal) variables x and the dual variables λ (representing the Lagrange multipliers) are treated as independent. The usual motivation f... |

164 |
Pathways to the optimal set in linear programming
- Megiddo
- 1989
(Show Context)
Citation Context ...surveys [65,99,101]). The evident similarity of interior methods to longstanding continuation approaches (see,e.g.,[1,2]) has been recognized since the early days of modern interior methods (see,e.g.,=-=[71]-=-),but numerous aspects remain to be explored. As a remarkable bonus,interior methods are playing a growing role in the study of hard combinatorial problems. Many of the most important problems in disc... |

143 |
Convex Analysis and Nonlinear Optimization
- Borwein, Lewis
- 2000
(Show Context)
Citation Context ... KKT conditions (2.2a)–(2.2d) are not enough to ensure optimality unless problem (2.1) is a convex program (Definition A.8). (For discussions of convex programming and convex analysis,see,for example,=-=[12,89]-=-.) Consequently,it is common to consider second-order optimality conditions as well. In this regard,the Hessian of the Lagrangian L(x, λ) (2.3) with respect to x will be crucial: (2.11) H(x, λ) △ = ∇ ... |

130 |
Path-following methods for linear programming
- Gonzaga
- 1992
(Show Context)
Citation Context ... the generality of the functions involved. For the case of linear programming, the central path is ultimately smooth and its approach to the optimal face is now completely understood (see,for example,=-=[55]-=-). For convex problems satisfying the LICQ,the barrier trajectory converges to the analytic center of the set of optimal solutions [70],but the very recent examples of [48] show that the central path ... |

126 |
Trust-region methods
- Conn, Gould, et al.
- 2000
(Show Context)
Citation Context ... the trust-region constraint (which,as its name suggests,specifies the local region in which the Taylor-series model can be trusted). See,e.g.,[28,34,52,77,80] for material on line-search methods,and =-=[25,28,34,80]-=- for discussion of trust-region methods. The presentation in section 3.3 focused on unconstrained minimizers of the log barrier function; hence we consider how those points might be found. 4.2. The Cl... |

124 | Sequential Quadratic Programming
- Boggs, Tolle
- 1995
(Show Context)
Citation Context ...ing mixed-constraint problems; here we consider how SQP might be applied to the all-equality problem (6.5). We emphasize that we address only a minute fraction of what can be said about SQP (see,e.g.,=-=[6,10,34,52,77,80]-=-) and that we do not exploit the full generality of SQP methods. SQP methods are based directly on optimality conditions,and accordingly they maintain estimates of both x and λ. The step p in the x-va... |

115 | A trust region method based on interior point techniques for nonlinear programming
- Byrd, Gilbert, et al.
- 1996
(Show Context)
Citation Context ...s in the classical primal barrier method,primal-dual methods based on properties of x(µ) are increasingly popular for solving general nonlinear programming problems; see,for example,the recent papers =-=[14,24,31,39,46,93,97,103]-=-. As in primal-dual methods for linear programming,the original (primal) variables x and the dual variables λ (representing the Lagrange multipliers) are treated as independent. The usual motivation f... |

108 |
The Fritz John necessary optimality conditions in the presence of equality constraints
- Mangasarian, Fromovitz
- 1967
(Show Context)
Citation Context ...t ¯ λ is unique,so that Mλ(¯x) contains a single multiplier when the LICQ holds at a KKT point. The second constraint qualification to be considered,the Mangasarian–Fromovitz constraint qualification =-=[67]-=-,ensures the existence of a direction along which,to first order,all active constraints strictly increase. Definition 2.10 (Mangasarian–Fromovitz constraint qualification). Consider an inequality-cons... |

98 | On the formulation and theory of Newton interior-point method for nonlinear programming
- El-Bakry, Tapia, et al.
- 1996
(Show Context)
Citation Context ...s in the classical primal barrier method,primal-dual methods based on properties of x(µ) are increasingly popular for solving general nonlinear programming problems; see,for example,the recent papers =-=[14,24,31,39,46,93,97,103]-=-. As in primal-dual methods for linear programming,the original (primal) variables x and the dual variables λ (representing the Lagrange multipliers) are treated as independent. The usual motivation f... |

97 | Interior-point methods for nonconvex nonlinear programming: Filter methods and merit functions
- Benson, Vanderbei, et al.
(Show Context)
Citation Context ...define a convergent filter algorithm; we refer the interested reader to [35,36]. Recently,several algorithms were proposed that use a filter to force convergence of a primal-dual interior method (see =-=[8,100,107]-=-). In the primal-dual context,there are several ways to define the filter pair,including the norm of the gradient of the Lagrangian and the proximity norm �C(x)λ − µe�. 6. Treatment of Equality Constr... |

95 | Eigenvalue optimization
- Lewis, Overton
- 1996
(Show Context)
Citation Context ... subject to semidefiniteness constraints—is arguably the most notable of these problems to receive widespread attention as a direct result of the development of interior methods (see,e.g.,the surveys =-=[65,99,101]-=-). The evident similarity of interior methods to longstanding continuation approaches (see,e.g.,[1,2]) has been recognized since the early days of modern interior methods (see,e.g.,[71]),but numerous ... |

95 |
Some NP-complete problems in quadratic and nonlinear programming
- Murty, Kabadi
- 1987
(Show Context)
Citation Context ...ion of a linear program (2.10); verifying assumption (iii) for all p such that J ∗ A p ≥ 0 requires finding the global minimizer of a possibly indefinite quadratic form over a cone,an NP-hard problem =-=[75,82]-=-,not to mention the issue of how to check that (iii) holds for all λ ∈Mλ. If,however,the gradients of the active constraints at x ∗ are linearly independent and strict complementarity holds,Theorem 2.... |

93 | Line search algorithms with guaranteed suf decrease
- More, Thuente
- 1994
(Show Context)
Citation Context ...t is straightforward to reduce α in a linesearch method until x + αp is strictly feasible. But since most general-purpose linesearch methods rely on quadratic,inverse quadratic,or cubic interpolation =-=[72]-=-,they may not be well suited to the extreme behavior of barrier functions,e.g.,the sharp rise near the boundary. Furthermore,a highly nonlinear constraint can be violated at points lying between two s... |

75 | Numerical Path Following
- Allgower, Georg
- 1997
(Show Context)
Citation Context ...ad attention as a direct result of the development of interior methods (see,e.g.,the surveys [65,99,101]). The evident similarity of interior methods to longstanding continuation approaches (see,e.g.,=-=[1,2]-=-) has been recognized since the early days of modern interior methods (see,e.g.,[71]),but numerous aspects remain to be explored. As a remarkable bonus,interior methods are playing a growing role in t... |

70 |
Accurate symmetric indefinite linear equation solvers
- Grimes, Ashcraft, et al.
- 1998
(Show Context)
Citation Context ...n is P T KP = LBL T ,sINTERIOR METHODS 577 where B is now a block-diagonal matrix with diagonal blocks of order 1 × 1or2× 2. Efficient sparse-matrix methods are also available for this case (see,e.g.,=-=[5,29]-=-),but additional overhead is likely since the choice of P depends on numerical considerations unrelated to the sparsity of the factors. In the neighborhood of a trajectory of minimizers we can expect ... |

66 |
A primal-dual interior method for nonconvex nonlinear programming
- Gay, Overton, et al.
- 1998
(Show Context)
Citation Context |

63 | Primal-dual interior methods for nonconvex nonlinear programming
- Forsgren, Gill
- 1998
(Show Context)
Citation Context |

63 | Symmetric quasi-definite matrices
- Vanderbei
- 1991
(Show Context)
Citation Context ...rix K of (5.6) is symmetric quasi-definite,which means that for every row and column permutation P there exists a factorization P T KP = LBL T ,where L is unit lower-triangular and B is diagonal (see =-=[53,102]-=-). This allows P to be selected solely for the purposes of obtaining a sparse factor L. For nonconvex problems,the relevant factorization is P T KP = LBL T ,sINTERIOR METHODS 577 where B is now a bloc... |

59 |
1999], An interior point algorithm for large-scale nonlinear programming
- Byrd, Hribar, et al.
(Show Context)
Citation Context ...ation (6.6) were applied directly to (6.9). Interior-point algorithms based on the quadratic programming subproblem (6.17) typically utilize SQP-based merit functions to enforce convergence (see,e.g.,=-=[15,27,46,83,123]-=-). If indefiniteness is addressed with a suitable line-search or trust-region technique,the limit points of the sequence of iterates satisfy the second-order necessary optimality conditions for (6.9) ... |

54 |
The logarithmic potential method of convex programming
- Frisch
- 1955
(Show Context)
Citation Context ...3.2) Ilog(x) △ = − m� ln ci(x), also appears to satisfy the desired properties. The earliest history of the logarithmic interior function is worth noting. Frisch’s 1955 “logarithmic potential method” =-=[44]-=- is based on using the gradient of f(x)+ �m i=1 αi ln ci(x) to retain feasibility and accelerate convergence; however,Frisch’s approach did not involve unconstrained minimization of this function. See... |

50 |
Modified barrier functions (theory and methods
- Polyak
- 1992
(Show Context)
Citation Context ...trained minimizers of the classical log barrier function converge to a solution of the original constrained problem only if the barrier parameter µ goes to zero. By contrast, modified barrier methods =-=[13, 23,54,76,84]-=- define a sequence of unconstrained problems in which the value of µ remains bounded away from zero,thereby avoiding the need to solve a problem whose Hessian becomes increasingly ill-conditioned as µ... |

49 |
An analogue of Moreau’s proximation theorem, with application to the nonlinear complementarity problem
- McLinden
- 1980
(Show Context)
Citation Context ...o the optimal face is now completely understood (see,for example,[55]). For convex problems satisfying the LICQ,the barrier trajectory converges to the analytic center of the set of optimal solutions =-=[70]-=-,but the very recent examples of [48] show that the central path can behave in strange and unexpected ways even for infinitely smooth convex functions. Despite these caveats,the “centered” nature of t... |

48 | Superlinear convergence of a stabilized SQP method to a degenerate solution
- Wright
- 1998
(Show Context)
Citation Context ...f the Jacobian J E is rank-deficient. Consequently,one needs to modify only H to make the matrix in (6.24) nonsingular [39]. (A related regularization approach for general SQP methods is described in =-=[117]-=-.) We began this section with a discussion of unconstrained minimization of the classical penalty-barrier function Φ PB (6.20),but the techniques presented have all beens586 ANDERS FORSGREN, PHILIP E.... |

42 | On practical conditions for the existence and uniqueness of solutions to the general equality quadratic programming problem - Gould - 1985 |

40 |
The factorization of sparse symmetric indefinite matrices
- Duff, Gould, et al.
- 1989
(Show Context)
Citation Context ...n is P T KP = LBL T ,sINTERIOR METHODS 577 where B is now a block-diagonal matrix with diagonal blocks of order 1 × 1or2× 2. Efficient sparse-matrix methods are also available for this case (see,e.g.,=-=[5,29]-=-),but additional overhead is likely since the choice of P depends on numerical considerations unrelated to the sparsity of the factors. In the neighborhood of a trajectory of minimizers we can expect ... |

40 | A globally convergent primal-dual interiorpoint filter method for nonconvex nonlinear programming
- Ulbrich, Ulbrich, et al.
(Show Context)
Citation Context ...define a convergent filter algorithm; we refer the interested reader to [35,36]. Recently,several algorithms were proposed that use a filter to force convergence of a primal-dual interior method (see =-=[8,100,107]-=-). In the primal-dual context,there are several ways to define the filter pair,including the norm of the gradient of the Lagrangian and the proximity norm �C(x)λ − µe�. 6. Treatment of Equality Constr... |

38 | Trust-region interior-point SQP algorithm for a class of nonlinear programming problems
- jr, Heinkenschloss, et al.
- 1998
(Show Context)
Citation Context ...ation (6.6) were applied directly to (6.9). Interior-point algorithms based on the quadratic programming subproblem (6.17) typically utilize SQP-based merit functions to enforce convergence (see,e.g.,=-=[15,27,46,83,123]-=-). If indefiniteness is addressed with a suitable line-search or trust-region technique,the limit points of the sequence of iterates satisfy the second-order necessary optimality conditions for (6.9) ... |

37 |
A necessary and sufficient regularity condition to have bounded multipliers in non convex programming
- Gauvin
- 1977
(Show Context)
Citation Context ...trictly feasible,but nonlinear, paths emanating from the origin. An important consequence of the MFCQ is that its satisfaction at a first-order KKT point implies boundedness of the set of multipliers =-=[45]-=-. Lemma 2.11 (implication 1 of MFCQ: a bounded multiplier set). If ¯x is a firstorder KKT point at which the MFCQ is satisfied, then the set of multipliers Mλ defined in (2.5) is bounded. Proof. First... |

36 | Failure of global convergence for a class of interior point methods for nonlinear programming
- Wächter, Biegler
(Show Context)
Citation Context ...x case is thus how to treat indefiniteness,a topic that would consume too many pages to allow its detailed treatment in this article. A more fundamental difficulty arises at the theoretical level. In =-=[106]-=-,a class of one-dimensional examples was recently defined for which certain line-search barrierSQP methods (section 6.2.2) fail to find a feasible point and,much more seriously,do not even converge to... |

35 |
On the average number of steps of the simplex method of linear programming
- Smale
- 1983
(Show Context)
Citation Context ...mial-time algorithms for linear programming. Furthermore,barrier methods (unlike the simplex method) could be applied not only to linear programming 1 Assuming various distributions of random inputs, =-=[11, 94]-=- showed that the simplex method converges in expected polynomial time. The recent development of “smoothed” complexity analysis [95] has led to new insights about the average behavior of the simplex m... |

34 |
Checking local optimality in constrained quadratic programming is NP-hard
- Pardalos, Schnitger
- 1988
(Show Context)
Citation Context ...ion of a linear program (2.10); verifying assumption (iii) for all p such that J ∗ A p ≥ 0 requires finding the global minimizer of a possibly indefinite quadratic form over a cone,an NP-hard problem =-=[75,82]-=-,not to mention the issue of how to check that (iii) holds for all λ ∈Mλ. If,however,the gradients of the active constraints at x ∗ are linearly independent and strict complementarity holds,Theorem 2.... |

33 |
Interior-point methods for convex programming
- Jarre
- 1992
(Show Context)
Citation Context ...of augmented Lagrangian methods for nonlinearly constrained optimization. We consider only shifts of the form (6.26b); for more general discussions of shifted constraints in interior methods,see,e.g.,=-=[23,46,60,84,98]-=- and section 4.6. The appeal of altering the constraints via either slacks or shifts may not be evident at first,since the dimension of the problem appears to have increased. However,two major benefit... |

32 |
On the accurate determination of search directions for simple differentiable penalty functions
- Gould
- 1986
(Show Context)
Citation Context ...ached. Instead of solving the classicalsINTERIOR METHODS 571 Newton system directly,it is possible to define an augmented or stretched system that is not inevitably ill-conditioned as µ → 0 (see,e.g.,=-=[57]-=-). Recalling the definition (4.20) H � x, π(x, µ) � = ∇ 2 f(x) − m� πi(x, µ)∇ 2 ci(x), with π(x, µ) =µ ·/ c(x), i=1 and using block elimination,it is easy to see that the Newton barrier direction p is... |

30 | Global convergence of trust-region SQP-filter algorithms for general nonlinear programming
- Fletcher, Gould, et al.
- 2002
(Show Context)
Citation Context ...d to the filter,and all filter entries that it dominates are removed. In practice many details need to be specified in order to define a convergent filter algorithm; we refer the interested reader to =-=[35,36]-=-. Recently,several algorithms were proposed that use a filter to force convergence of a primal-dual interior method (see [8,100,107]). In the primal-dual context,there are several ways to define the f... |

30 |
Solving symmetric indefinite systems in an interior-point method for linear programming
- Pourer, Mehrotra
- 1993
(Show Context)
Citation Context ...HC (except when the constraints are simple bounds,in which case HC is simply H plus a positive diagonal matrix). An alternative strategy is to factorize the full (n+m)×(n+m) system in (5.2) (see,e.g.,=-=[43,49,50]-=-),typically after symmetrizing the system. A symmetric matrix can be created by multiplying the second block of equations in (5.2) by Λ−1 and changing the sign of the second block of columns,giving (5... |

30 |
Ill-conditioning and computational error in interior methods for nonlinear programming
- Wright
- 1998
(Show Context)
Citation Context ... has been repeatedly observed that the computed solutions in interior methods are almost always much more accurate than they should be (see,for example, [90,110]). To explain why,numerous papers,e.g.,=-=[41,85,113,114,116,118,119]-=-, have examined ill-conditioning in several contexts,including linear programming,linear complementarity,and primal-dual systems for nonconvex nonlinear optimization. One of the surprising results tha... |