## An interior algorithm for nonlinear optimization that combines line search and trust region steps (2006)

### Cached

### Download Links

- [www.ece.northwestern.edu]
- [users.eecs.northwestern.edu]
- [www.ece.northwestern.edu]
- DBLP

### Other Repositories/Bibliography

Venue: | Mathematical Programming 107 |

Citations: | 31 - 11 self |

### BibTeX

@TECHREPORT{Waltz06aninterior,

author = {R. A. Waltz and J. L. Morales and J. Nocedal and D. Orban},

title = {An interior algorithm for nonlinear optimization that combines line search and trust region steps},

institution = {Mathematical Programming 107},

year = {2006}

}

### OpenURL

### Abstract

An interior-point method for nonlinear programming is presented. It enjoys the flexibility of switching between a line search method that computes steps by factoring the primal-dual equations and a trust region method that uses a conjugate gradient iteration. Steps computed by direct factorization are always tried first, but if they are deemed ineffective, a trust region iteration that guarantees progress toward stationarity is invoked. To demonstrate its effectiveness, the algorithm is implemented in the Knitro [6, 28] software package and is extensively tested on a wide selection of test problems. 1

### Citations

1914 | Numerical optimization - Nocedal, Wright |

1082 |
Practical Methods of Optimization
- Fletcher
- 1987
(Show Context)
Citation Context ... the solution, a phenomenon known as the Maratos effect. This deficiency can be overcome by applying a second-order correction step (SOC), which is a Newton-like step that aims to improve feasibility =-=[14]-=-. We apply the second-order correction when the first trial steplength α T = 1 is rejected and if the reason for the rejection can be attributed solely to an increase in the norm of the constraints, t... |

246 | Benchmarking optimization software with performance profiles
- Dolan, Moré
(Show Context)
Citation Context ...14c) were set as ɛ opt = ɛ feas = 10 −6 . For details on the convergence criteria used in Knitro 3.0, see [28]. We report results using the logarithmic performance profiles proposed by Dolan and Moré =-=[10]-=-. Let tp,s denote the time to solve problem p by solver s. We define the ratios rp,s = tp,s t∗ , (4.25) p where t ∗ p is the lowest time required by any code to solve problem p. If a code does not sol... |

160 | Nonlinear programming without a penalty function
- Fletcher, Leyffer
(Show Context)
Citation Context ...ctorization of the KKT matrix [1]. Other approaches include the use of ℓ1 or ℓ2 penalizations of the constraints, which provide regularization [16, 24], and the use of a feasibility restoration phase =-=[15, 27]-=-. In this paper we describe a mechanism for stabilizing the line search iteration that is different from those proposed in the literature. It consists of falling back, under certain conditions, on a t... |

145 | An interior-point algorithm for nonconvex nonlinear programming
- Vanderbei, Shanno
- 1999
(Show Context)
Citation Context ...t. h(x) = 0 (1.1b) g(x) ≤ 0, (1.1c) where f : R n → R, h : R n → R l and g : R n → R m are twice continuously differentiable functions. A variety of line search interior algorithms have been proposed =-=[12, 16, 25, 27, 30]-=-, several of which have been implemented in high-quality software packages [2, 25, 27]. The search direction is computed in these algorithms by factoring the primal-dual system. In order to achieve ro... |

126 |
The conjugate gradient method and trust regions in large scale optimization
- Steihaug
- 1983
(Show Context)
Citation Context ...too expensive, and one should resort to the computation of Cauchy steps only when needed. Second, a dog-leg approach is not well defined in the case of negative curvature, where a Newton-CG iteration =-=[23]-=- is more appropriate. These observations and the fact that the first release of Knitro implements a Newton-CG iteration motivated us to follow the approach just outlined. Many details of Algorithm 2.1... |

123 |
CUTEr, a Constrained and Unconstrained Test Environment, revisited. http://cuter.rl.ac.uk/cuter-www
- Toint
(Show Context)
Citation Context ... the majority of the iterations are obtained by factoring the primal-dual system. We compare it with the trust region algorithm in Knitro 3.0, henceforth called Knitro-CG. We use the CUTEr collection =-=[3, 18]-=- as of June 5, 2003, from which 968 problems have been retained; the remaining problems have been discarded because they require too much memory. All tests were performed on a 2.8 GHz Pentium Xeon, wi... |

105 | A trust region method based on interior point techniques for nonlinear programming
- Byrd, Gilbert, et al.
- 2000
(Show Context)
Citation Context ...ration (2.4)-(2.7) and replace it with a trust region step. The resulting algorithm possesses global convergence properties similar to those of the algorithms implemented in FilterSQP [15] and Knitro =-=[5]-=-. We outline the method in Algorithm 2.1. There φν(z) denotes a merit function using a penalty parameter ν, and Dφν(z; dz) denotes the directional derivative of φν along a direction dz. An inertia-rev... |

103 | Representations of quasi-Newton matrices and their use in limited memory methods
- Byrd, Nocedal, et al.
- 1994
(Show Context)
Citation Context ...desirable to use limited memory updating to avoid the storage and manipulation of a dense n × n matrix. We have implemented a limited memory BFGS method using the compact representations described in =-=[8]-=-. Here B has the form B = ξI + NMN T , (3.23) where ξ > 0 is a scaling factor, N is an n × 2p matrix, M is a 2p × 2p symmetric and nonsingular matrix, and p denotes the number of correction pairs save... |

103 |
On the implementation of a primal-dual interior point filter line search algorithm for large-scale nonlinear programming
- Wächter, Biegler
- 2006
(Show Context)
Citation Context ...t. h(x) = 0 (1.1b) g(x) ≤ 0, (1.1c) where f : R n → R, h : R n → R l and g : R n → R m are twice continuously differentiable functions. A variety of line search interior algorithms have been proposed =-=[12, 16, 25, 27, 30]-=-, several of which have been implemented in high-quality software packages [2, 25, 27]. The search direction is computed in these algorithms by factoring the primal-dual system. In order to achieve ro... |

87 |
Direct methods for solving symmetric indefinite systems of linear equations
- Bunch, Parlett
- 1971
(Show Context)
Citation Context ...There φν(z) denotes a merit function using a penalty parameter ν, and Dφν(z; dz) denotes the directional derivative of φν along a direction dz. An inertia-revealing symmetric indefinite factorization =-=[4]-=- of the primal-dual matrix in (2.4) provides its number of negative eigenvalues. If this number exceeds l + m, then dz cannot be guaranteed to be a descent direction (see, e.g., [21, Lemma 16.3]), and... |

82 |
On the formulation and theory of the Newton interior-point method for nonlinear programming
- El-Bakry, Tapia, et al.
- 1996
(Show Context)
Citation Context ...t. h(x) = 0 (1.1b) g(x) ≤ 0, (1.1c) where f : R n → R, h : R n → R l and g : R n → R m are twice continuously differentiable functions. A variety of line search interior algorithms have been proposed =-=[12, 16, 25, 27, 30]-=-, several of which have been implemented in high-quality software packages [2, 25, 27]. The search direction is computed in these algorithms by factoring the primal-dual system. In order to achieve ro... |

72 | An interior point algorithm for large scale nonlinear programming
- Byrd, Hribar, et al.
- 1999
(Show Context)
Citation Context ...rst, but if they are deemed ineffective, a trust region iteration that guarantees progress toward stationarity is invoked. To demonstrate its effectiveness, the algorithm is implemented in the Knitro =-=[6, 28]-=- software package and is extensively tested on a wide selection of test problems. 1 Introduction In this paper we describe an interior method for nonlinear programming and discuss its software impleme... |

68 | Implementation of interior point methods for large scale linear programming
- Andersen, Gondzio, et al.
- 1996
(Show Context)
Citation Context ...lementation. The difficulties caused by rank deficient constraint Jacobians are sometimes addressed at the linear algebra level by introducing perturbations during the factorization of the KKT matrix =-=[1]-=-. Other approaches include the use of ℓ1 or ℓ2 penalizations of the constraints, which provide regularization [16, 24], and the use of a feasibility restoration phase [15, 27]. In this paper we descri... |

61 |
Trust-region methods
- Toint
- 2000
(Show Context)
Citation Context ... 0 0 otherwise. (3.3) Here W stands for W (z, λ; µ) and is defined in (2.6). If σ = 1, pred is the standard quadratic/linear model of the merit function used in a variety of trust region methods; see =-=[9]-=-. We allow σ to have the value zero because, as we argue below, including the term d T z W dz in (3.2) when it is negative could cause the algorithm to fail. Following [6, 13], we choose the penalty p... |

58 | Primal-dual interior methods for nonconvex nonlinear programming
- Forsgren, Gill
- 1998
(Show Context)
Citation Context |

39 |
CUTEr (and SifDec), a Constrained and Unconstrained Testing Environment, revisited
- Toint
- 2003
(Show Context)
Citation Context ... the majority of the iterations are obtained by factoring the primal-dual system. We compare it with the trust region algorithm in Knitro 3.0, henceforth called Knitro-CG. We use the CUTEr collection =-=[3, 18]-=- as of June 5, 2003, from which 968 problems have been retained; the remaining problems have been discarded because they require too much memory. All tests were performed on a 2.8 GHz Pentium Xeon, wi... |

34 | Failure of Global Convergence for a Class of Interior Point Methods for Nonlinear Programming, Mathematical Programming 88(3), p. 565574
- Wächter, Biegler
- 2000
(Show Context)
Citation Context ....7) provides the basis for most line search interior methods. This remarkably simple approach must, however, be modified to cope with non-convexity and to prevent convergence to non-stationary points =-=[7, 26]-=-. Instead of modifying the primaldual matrix, as is commonly done, we use a safeguarding trust region step to stabilize the iteration, for two reasons. First, when W is not positive definite on the nu... |

30 | A primal-dual interior-point method for nonlinear programming with strong global and local convergence properties
- Tits, Wächter, et al.
- 2003
(Show Context)
Citation Context ... algebra level by introducing perturbations during the factorization of the KKT matrix [1]. Other approaches include the use of ℓ1 or ℓ2 penalizations of the constraints, which provide regularization =-=[16, 24]-=-, and the use of a feasibility restoration phase [15, 27]. In this paper we describe a mechanism for stabilizing the line search iteration that is different from those proposed in the literature. It c... |

26 |
GALAHAD, a library of thread-safe Fortran 90 packages for large-scale nonlinear optimization
- Toint
(Show Context)
Citation Context ...l component of the step is obtained by using a projected Krylov iteration, as is done with conjugate gradients in the Knitro package [28], or using a Lanczos method, as is done in the GALAHAD package =-=[19]-=-. Second, we would like to take advantage of the robustness of trust region steps in the presence of Hessian or Jacobian rank deficiencies. We have in mind trust region methods that provide Cauchy dec... |

26 |
A globally convergent primaldual interior point method for constrained optimization
- Yamashita
- 1992
(Show Context)
Citation Context |

20 |
Numerical Stability and Efficiency of Penalty Algorithms
- Dussault
- 1995
(Show Context)
Citation Context ... 3.5 Update of µ and Barrier Stopping Test The sequence of barrier parameters {µk} must converge to zero, and should do so quickly if possible. Superlinear rules for decreasing µ have been studied in =-=[11, 17, 22, 29]-=-, but they employ various parameters that can be difficult to select in practice. We use instead the following simple strategy for updating µ that has performed as well in our tests as have more compl... |

16 |
Superlinear Convergence of Primal-Dual Interior Point Algorithms for Nonlinear Programming
- Toint
(Show Context)
Citation Context ... 3.5 Update of µ and Barrier Stopping Test The sequence of barrier parameters {µk} must converge to zero, and should do so quickly if possible. Superlinear rules for decreasing µ have been studied in =-=[11, 17, 22, 29]-=-, but they employ various parameters that can be difficult to select in practice. We use instead the following simple strategy for updating µ that has performed as well in our tests as have more compl... |

13 |
and J.Nocedal, “Knitro user’s manual
- Waltz
- 2003
(Show Context)
Citation Context ...rst, but if they are deemed ineffective, a trust region iteration that guarantees progress toward stationarity is invoked. To demonstrate its effectiveness, the algorithm is implemented in the Knitro =-=[6, 28]-=- software package and is extensively tested on a wide selection of test problems. 1 Introduction In this paper we describe an interior method for nonlinear programming and discuss its software impleme... |

9 | On the Convergence of Newton Iterations to Non-Stationary - Byrd, Marazzi, et al. - 2001 |

9 | Q-superlinear convergence of the iterates in primal-dual interior-point methods
- Potra
- 2001
(Show Context)
Citation Context ... 3.5 Update of µ and Barrier Stopping Test The sequence of barrier parameters {µk} must converge to zero, and should do so quickly if possible. Superlinear rules for decreasing µ have been studied in =-=[11, 17, 22, 29]-=-, but they employ various parameters that can be difficult to select in practice. We use instead the following simple strategy for updating µ that has performed as well in our tests as have more compl... |

9 | Q-superlinear convergence of primal-dual interior point quasi-Newton methods for constrained optimization - Yabe, Yamashita |

8 |
A catalogue of subroutines (HSL
- Library
- 2002
(Show Context)
Citation Context ...lations in the size of ∆k. 3.7 Solution of the Primal-Dual Equations The primal-dual system (2.4) is solved by using the symmetric indefinite factorization implemented in the HSL library routine MA27 =-=[20]-=-. This routine provides the number of negative eigenvalues, neig, of the primal-dual system. An important practical consideration is the choice of the pivot tolerance. We set it initially to 10 −8 bec... |

5 |
An interior-point nonlinear programming algorithm for large scale optimization
- Betts, Eldersveld, et al.
- 2003
(Show Context)
Citation Context ...e continuously differentiable functions. A variety of line search interior algorithms have been proposed [12, 16, 25, 27, 30], several of which have been implemented in high-quality software packages =-=[2, 25, 27]-=-. The search direction is computed in these algorithms by factoring the primal-dual system. In order to achieve robustness, these line search approaches must successfully address two issues: • How to ... |

5 |
A hybrid algorithm for nonlinear equality constrained optimization problems: global and local convergence theory
- El-Hallabi
- 1999
(Show Context)
Citation Context ...given by (2.5), and ν > 0 is the penalty parameter, which is updated at each iteration so that the search direction dz given by (2.4) is a descent direction for φν. Our update rule for ν, proposed in =-=[13]-=-, is inspired by trust region methods. Instead of requiring only that the directional derivative of φν be negative, as is commonly done, we choose ν based on the decrease in a quadratic/linear model o... |