## The interior-point revolution in optimization: history, recent developments, and lasting consequences (2005)

Venue: | Bull. Amer. Math. Soc. (N.S |

Citations: | 19 - 1 self |

### BibTeX

@ARTICLE{Wright05theinterior-point,

author = {Margaret H. Wright},

title = {The interior-point revolution in optimization: history, recent developments, and lasting consequences},

journal = {Bull. Amer. Math. Soc. (N.S},

year = {2005},

volume = {42},

pages = {39--56}

}

### OpenURL

### Abstract

Abstract. Interior methods are a pervasive feature of the optimization landscape today, but it was not always so. Although interior-point techniques, primarily in the form of barrier methods, were widely used during the 1960s for problems with nonlinear constraints, their use for the fundamental problem of linear programming was unthinkable because of the total dominance of the simplex method. During the 1970s, barrier methods were superseded, nearly to the point of oblivion, by newly emerging and seemingly more efficient alternatives such as augmented Lagrangian and sequential quadratic programming methods. By the early 1980s, barrier methods were almost universally regarded as a closed chapter in the history of optimization. This picture changed dramatically in 1984, when Narendra Karmarkar announced a fast polynomial-time interior method for linear programming; in 1985, a formal connection was established between his method and classical barrier methods. Since then, interior methods have continued to transform both the theory and practice of constrained optimization. We present a condensed,

### Citations

2248 |
Numerical optimization
- Nocedal, Wright
- 2000
(Show Context)
Citation Context ...asures progress. In unconstrained optimization, the merit function is typically the objective function. Standard line search acceptance criteria that ensure convergence are discussed in, for example, =-=[29, 28]-=-. A second strategy is based on defining a trust region around the current iterate within which the local model can be trusted. In optimization, the step in a trust-region method is typically chosen t... |

1571 |
Theory of linear and integer programming
- Schrijver
- 1986
(Show Context)
Citation Context ... A fundamental property of linear programs is that, if the optimal objective value is finite, a vertex minimizer must exist. (For details about linear programming and its terminology, see, e.g., [5], =-=[31]-=-, and [19].) The simplex method, invented by George B. Dantzig in 1947, is an iterative procedure for solving LPs that completely depends on this property. The starting point for the simplex method mu... |

1569 |
Practical Optimization
- Gill, Murray, et al.
(Show Context)
Citation Context ...ian function. Augmented Lagrangian methods and sequential quadratic programming (SQP) methods became especially popular and remain so today. For further details about these methods, see, for example, =-=[16, 10, 28]-=-. 3. The revolution begins 3.1. Karmarkar’s method. In 1984, Narendra Karmarkar [21] announced a polynomial-time LP method for which he reported solution times that were consistently 50 times faster t... |

973 | Improved Approximation Algorithms for Maximum Cut and Satisfiability Problems Using Semidefinite Programming
- Goemans, Williamson
- 1995
(Show Context)
Citation Context ...ted NP-hard problem. For example, a semidefinite program formulation leads to an approximate solution of the max-cut problem whose objective value is within a factor of 1.14 of the optimal value; see =-=[18]-=-. This kind of relationship guarantees that good approximate solutions to NP-hard problems can be computed in polynomial time. Interior methods are important in system and control theory because of th... |

851 | Semidefinite programming
- Vandenberghe, Boyd
- 1996
(Show Context)
Citation Context ... primal-dual equations (29) in linear programming. Semidefinite programming is an extremely lively research area today, producing new theory, algorithms, and implementations; see the surveys [33] and =-=[34]-=-. 5.2. New applications of interior methods. Interior methods are playing major roles in at least two areas: approximation techniques for NP-hard combinatorial problems, and system and control theory.... |

695 | A new polynomial-time algorithm for linear programming
- Karmarkar
- 1984
(Show Context)
Citation Context ...s became especially popular and remain so today. For further details about these methods, see, for example, [16, 10, 28]. 3. The revolution begins 3.1. Karmarkar’s method. In 1984, Narendra Karmarkar =-=[21]-=- announced a polynomial-time LP method for which he reported solution times that were consistently 50 times faster than the simplex method. This event, which received publicity around the world throug... |

595 |
Rheinboldt, Iterative Solution of Nonlinear Equations
- Ortega, C
- 1970
(Show Context)
Citation Context ...asures progress. In unconstrained optimization, the merit function is typically the objective function. Standard line search acceptance criteria that ensure convergence are discussed in, for example, =-=[29, 28]-=-. A second strategy is based on defining a trust region around the current iterate within which the local model can be trusted. In optimization, the step in a trust-region method is typically chosen t... |

499 | Primal-dual interior-point methods for semidefinite programming: Stability, convergence, and numerical results
- Alizadeh, Haeberly, et al.
- 1998
(Show Context)
Citation Context ...1 with C − � m i=1 yiAi � 0. Newton’s method cannot be applied directly to solve (37) and (41) because the matrix on the left-hand side of (41) is not symmetric. A primal approach, first suggested in =-=[1]-=-, is to replace (41) by the relation X(C − � yiAi)+(C − � yiAi)X =2µI. An analogous primal-dual method, called the “XZ + ZX method” for obvious reasons, is defined by finding (Xµ,yµ,Zµ), where Xµ ≻ 0a... |

356 |
Nonlinear Programming: Sequential Unconstrained Minimization Techniques
- Fiacco, McCormick
- 1968
(Show Context)
Citation Context ...if cj(x) < 0for any j. (This behavior constitutes an obvious rationale for the descriptors “barrier” and “interior”.) Numerous properties of B(x, µ) are known; see, for example, the classic reference =-=[9]-=- or [36, 12]. For small µ, unconstrained minimizers of B(x, µ) are related in an intuitively appealing way to the solution x ∗ of (4). Given that x ∗ satisfies the sufficient optimality conditions giv... |

251 | Interior Point Algorithms: Theory and analysis
- Ye
- 1997
(Show Context)
Citation Context ...erties, such as perturbed complementarity (19). Readers interested in Karmarkar’s method should consult his original paper [21] or any of the many comprehensive treatments published since 1984 (e.g., =-=[19, 30, 41, 35, 44]-=-).sTHE INTERIOR-POINT REVOLUTION 47 Beyond establishing the formal connection between Karmarkar’s method and barrier methods, [15] reported computational results comparing a state-of-the-art (in 1985)... |

156 |
Practical Methods of Optimization, Second Edition
- Fletcher
- 2000
(Show Context)
Citation Context ...ian function. Augmented Lagrangian methods and sequential quadratic programming (SQP) methods became especially popular and remain so today. For further details about these methods, see, for example, =-=[16, 10, 28]-=-. 3. The revolution begins 3.1. Karmarkar’s method. In 1984, Narendra Karmarkar [21] announced a polynomial-time LP method for which he reported solution times that were consistently 50 times faster t... |

130 |
Path-following methods for linear programming
- Gonzaga
- 1992
(Show Context)
Citation Context ...h (barrier trajectory) for a standard-form LP is defined by vectors xµ and yµ satisfying Axµ = b, xµ > 0; A T yµ + µX −1 (25) µ 1 = c. The central path has numerous properties of interest; see, e.g., =-=[20, 19, 35, 41]-=-, and [44]. Assume that we are given a point x>0forwhichAx = b. Using (23), the Newton equations (10) for problem (22) are � µX (26) −2 AT �� � � p −c + µX = A 0 −y −1 � 1 , 0 so that the Newton step ... |

122 | Semidefinite Optimization
- Todd
- 2001
(Show Context)
Citation Context ...) and the primal-dual equations (29) in linear programming. Semidefinite programming is an extremely lively research area today, producing new theory, algorithms, and implementations; see the surveys =-=[33]-=- and [34]. 5.2. New applications of interior methods. Interior methods are playing major roles in at least two areas: approximation techniques for NP-hard combinatorial problems, and system and contro... |

115 | A trust region method based on interior point techniques for nonlinear programming
- Byrd, Gilbert, et al.
- 1996
(Show Context)
Citation Context ...rimal barrier methods (see Section 4.3), primal-dual methods based on properties of xµ are increasingly popular for solving general nonlinear programming problems; see, for example, the recent papers =-=[8, 4, 11, 7, 14]-=-. As in primal-dual methods for LP, the original (primal) variables x and the dual variables λ (representing the Lagrange multipliers) are treated as independent. The usual motivation for primal-dual ... |

105 |
Theory and algorithms for linear optimization: an interior point approach
- Vial
- 1997
(Show Context)
Citation Context ...erties, such as perturbed complementarity (19). Readers interested in Karmarkar’s method should consult his original paper [21] or any of the many comprehensive treatments published since 1984 (e.g., =-=[19, 30, 41, 35, 44]-=-).sTHE INTERIOR-POINT REVOLUTION 47 Beyond establishing the formal connection between Karmarkar’s method and barrier methods, [15] reported computational results comparing a state-of-the-art (in 1985)... |

98 | On the formulation and theory of Newton interior-point method for nonlinear programming
- El-Bakry, Tapia, et al.
- 1996
(Show Context)
Citation Context ...rimal barrier methods (see Section 4.3), primal-dual methods based on properties of xµ are increasingly popular for solving general nonlinear programming problems; see, for example, the recent papers =-=[8, 4, 11, 7, 14]-=-. As in primal-dual methods for LP, the original (primal) variables x and the dual variables λ (representing the Lagrange multipliers) are treated as independent. The usual motivation for primal-dual ... |

90 | Interior Methods for Nonlinear Optimization
- Forsgren, Gill, et al.
(Show Context)
Citation Context ...) < 0for any j. (This behavior constitutes an obvious rationale for the descriptors “barrier” and “interior”.) Numerous properties of B(x, µ) are known; see, for example, the classic reference [9] or =-=[36, 12]-=-. For small µ, unconstrained minimizers of B(x, µ) are related in an intuitively appealing way to the solution x ∗ of (4). Given that x ∗ satisfies the sufficient optimality conditions given in Sectio... |

75 | On projected Newton barrier methods for linear programming and an equivalence to Karmarkar’s projective method
- Gill, Murray, et al.
- 1986
(Show Context)
Citation Context ...e geometry was used in its description; and no information was available about the implementation. Amid the frenzy of interest in Karmarkar’s method, it was shown in 1985 (and published the next year =-=[15]-=-) that there was a formal equivalence between Karmarkar’s method and the classical logarithmic barrier method applied to the LP problem. Soon researchers began to view once-discarded barrier methods i... |

66 | Solving real-world linear programs: a decade and more of progress
- Bixby
(Show Context)
Citation Context ...eed limit. In LP today, interior methods are faster than simplex for some very large problems, the reverse is true for some problems, and the two approaches are more or less comparable on others; see =-=[2]-=-. Consequently, commercial LP codes routinely offer both options. Further analysis is still needed of the problem characteristics that determine which approach is more appropriate. Unless a drastic ch... |

66 |
A primal-dual interior method for nonconvex nonlinear programming
- Gay, Overton, et al.
- 1998
(Show Context)
Citation Context ...rimal barrier methods (see Section 4.3), primal-dual methods based on properties of xµ are increasingly popular for solving general nonlinear programming problems; see, for example, the recent papers =-=[8, 4, 11, 7, 14]-=-. As in primal-dual methods for LP, the original (primal) variables x and the dual variables λ (representing the Lagrange multipliers) are treated as independent. The usual motivation for primal-dual ... |

63 | Primal-dual interior methods for nonconvex nonlinear programming
- Forsgren, Gill
- 1998
(Show Context)
Citation Context ...ntial recent progress in understanding this issue. A detailed analysis was given in [37] of the structure of the primal barrier Hessian (16) in an entire neighborhood of the solution. Several papers (=-=[13, 11, 40, 42]-=-) have analyzed the stability of specific factorizations for various interior methods. Very recently, the (at first) surprising result was obtained ([39, 43]) that, under conditions normally holding i... |

30 |
Ill-conditioning and computational error in interior methods for nonlinear programming
- Wright
- 1998
(Show Context)
Citation Context ...d of the solution. Several papers ([13, 11, 40, 42]) have analyzed the stability of specific factorizations for various interior methods. Very recently, the (at first) surprising result was obtained (=-=[39, 43]-=-) that, under conditions normally holding in practice, ill-conditioning of certain key matrices in interior methods for nonlinear programming does not noticeably degrade the accuracy of the computed s... |

25 |
Introduction to Linear and Nonlinear
- Luenberger
- 1984
(Show Context)
Citation Context ...rkably, this separation was a fully accepted part of the culture of optimization—indeed, it was viewed by some as inherent and unavoidable. For example, in a widely used and highly respected textbook =-=[24]-=- published in 1973, the authorcommentsintheprefacethat “PartII [unconstrainedoptimization] ...is independent of Part I [linear programming]” and that “except in a few isolated sections, this part [con... |

25 | Modified Cholesky factorizations in interior-point algorithms for linear programming
- Wright
- 1996
(Show Context)
Citation Context ...ntial recent progress in understanding this issue. A detailed analysis was given in [37] of the structure of the primal barrier Hessian (16) in an entire neighborhood of the solution. Several papers (=-=[13, 11, 40, 42]-=-) have analyzed the stability of specific factorizations for various interior methods. Very recently, the (at first) surprising result was obtained ([39, 43]) that, under conditions normally holding i... |

23 | Why a pure primal Newton barrier step may be infeasible
- Wright
- 1995
(Show Context)
Citation Context ...fect accuracy, primal barriers50 MARGARET H. WRIGHT methods suffer from inherently poor scaling of the search direction during the early iterations following a reduction of the barrier parameter; see =-=[38, 6]-=-. Thus, unless special precautions are taken, a full Newton step cannot be taken immediately after the barrier parameter is reduced. This fundamentally undesirable property implies that the classical ... |

22 |
Analytic expression for the eigenvalues and eigenvectors of the Hessian matrices of barrier and penalty functions
- Murray
- 1971
(Show Context)
Citation Context ...or various reasons, including perceived inefficiency compared to alternative strategies and worries about inherent ill-conditioning. With respect to the latter, it was observed in the late 1960s (see =-=[23, 25]-=-) that, if 1 ≤ ˆm <n,thencondHB(xµ,µ)=Θ(1/µ), so that the barrier Hessian becomes arbitrarily ill-conditioned at points lying on the barrier trajectory as µ → 0. Although it is impossible after a gap ... |

19 |
Stability of symmetric ill-conditioned systems arising in interior methods for constrained optimization
- Forsgren, Gill, et al.
- 1996
(Show Context)
Citation Context ...ntial recent progress in understanding this issue. A detailed analysis was given in [37] of the structure of the primal barrier Hessian (16) in an entire neighborhood of the solution. Several papers (=-=[13, 11, 40, 42]-=-) have analyzed the stability of specific factorizations for various interior methods. Very recently, the (at first) surprising result was obtained ([39, 43]) that, under conditions normally holding i... |

17 | Stability of linear equations solvers in interior-point methods
- Wright
- 1995
(Show Context)
Citation Context |

16 |
Interior methods for constrained optimization, in Acta Numerica
- Wright
- 1992
(Show Context)
Citation Context ...) < 0for any j. (This behavior constitutes an obvious rationale for the descriptors “barrier” and “interior”.) Numerous properties of B(x, µ) are known; see, for example, the classic reference [9] or =-=[36, 12]-=-. For small µ, unconstrained minimizers of B(x, µ) are related in an intuitively appealing way to the solution x ∗ of (4). Given that x ∗ satisfies the sufficient optimality conditions given in Sectio... |

14 |
Some properties of the Hessian of the logarithmic barrier function
- Wright
- 1994
(Show Context)
Citation Context ... of ill-conditioning, as noted earlier, has haunted interior methods since the late 1960s, but there has been substantial recent progress in understanding this issue. A detailed analysis was given in =-=[37]-=- of the structure of the primal barrier Hessian (16) in an entire neighborhood of the solution. Several papers ([13, 11, 40, 42]) have analyzed the stability of specific factorizations for various int... |

13 |
Hessian matrices of penalty functions for solving constrainedoptimization problems
- Lootsma
- 1969
(Show Context)
Citation Context ...or various reasons, including perceived inefficiency compared to alternative strategies and worries about inherent ill-conditioning. With respect to the latter, it was observed in the late 1960s (see =-=[23, 25]-=-) that, if 1 ≤ ˆm <n,thencondHB(xµ,µ)=Θ(1/µ), so that the barrier Hessian becomes arbitrarily ill-conditioned at points lying on the barrier trajectory as µ → 0. Although it is impossible after a gap ... |

7 | A note on using alternative second-order models for the subproblems arising in barrier function methods for minimization, Numer
- Conn, Gould, et al.
- 1994
(Show Context)
Citation Context ...fect accuracy, primal barriers50 MARGARET H. WRIGHT methods suffer from inherently poor scaling of the search direction during the early iterations following a reduction of the barrier parameter; see =-=[38, 6]-=-. Thus, unless special precautions are taken, a full Newton step cannot be taken immediately after the barrier parameter is reduced. This fundamentally undesirable property implies that the classical ... |

7 | Effects of finite-precision arithmetic on interior-point methods for nonlinear programming
- Wright
(Show Context)
Citation Context ...d of the solution. Several papers ([13, 11, 40, 42]) have analyzed the stability of specific factorizations for various interior methods. Very recently, the (at first) surprising result was obtained (=-=[39, 43]-=-) that, under conditions normally holding in practice, ill-conditioning of certain key matrices in interior methods for nonlinear programming does not noticeably degrade the accuracy of the computed s... |

2 |
Toint (2000). A primal-dual trust-region algorithm for minimizing a non-convex function subject to bound and linear equality constraints
- Conn, Gould, et al.
(Show Context)
Citation Context |

2 |
Why the simplex method usually takes polynomial time
- Spielman, Teng
- 2004
(Show Context)
Citation Context ... (5) λ ∗ jcj (x∗ (6) ) = 0, j =1,...,m; 2 A side comment: Recent work on “smoothed complexity” provides a fascinating explanation of why the simplex method is usually a polynomial-time algorithm; see =-=[32]-=-.sTHE INTERIOR-POINT REVOLUTION 43 (3) λ ∗ j > 0ifcj(x ∗ )=0,j =1, ..., m; (4) N(x ∗ ) T W (x ∗ ,λ ∗ )N(x ∗ ), the reduced Hessian of the Lagrangian, is positive definite. Relation (6), that each pair... |