## Two-Step Algorithms for Nonlinear Optimization with Structured Applications (1999)

Venue: | SIAM Journal on Optimization |

Citations: | 10 - 6 self |

### BibTeX

@ARTICLE{Conn99two-stepalgorithms,

author = {Andrew R. Conn and Luís N. Vicente and Chandu Visweswariah},

title = {Two-Step Algorithms for Nonlinear Optimization with Structured Applications},

journal = {SIAM Journal on Optimization},

year = {1999},

volume = {9},

pages = {924--947}

}

### Years of Citing Articles

### OpenURL

### Abstract

In this paper we propose extensions to trust-region algorithms in which the classical step is augmented with a second step that we insist yields a decrease in the value of the objective function. The classical convergence theory for trust-region algorithms is adapted to this class of two-step algorithms. The algorithms can be applied to any problem with variable(s) whose contribution to the objective function is a known functional form. In the nonlinear programming package LANCELOT, they have been applied to update slack variables and variables introduced to solve minimax problems, leading to enhanced optimization eciency. Extensive numerical results are presented to show the eectiveness of these techniques. Keywords. Trust regions, line searches, two-step algorithms, spacer steps, slack variables, LANCELOT, minimax problems, expensive function evaluations, circuit optimization. AMS subject classications. 49M37, 90C06, 90C30 1 Introduction In nonlinear optimization proble...

### Citations

1085 | Practical methods of optimization - Fletcher - 1991 |

1046 |
Introduction to Linear and Nonlinear Programming
- Luenberger
- 1973
(Show Context)
Citation Context ...al criteria required for global convergence. We point out that the conservative two-step line-search algorithm is not new and can be found in the books by Bertsekas [1], Section 1.3.1, and Luenberger =-=[19]-=-, Section 7.10, where the second step is called a spacer step. A description of the greedy and conservative two-step line-search algorithms can be found in [11]. In trust regions, if the second step i... |

944 |
Numerical methods for unconstrained optimization and nonlinear equations
- Dennis, Schnabel
- 1983
(Show Context)
Citation Context ...nd set s k = s k + ^ s k . 2.3 Set y k+1 = y k + s k . The proof of the local convergence rate of the two-step Newton's method requires a few modications from the standard proof of Newton's method [1=-=2-=-], Theorem 5.2.1. Recall that that proof of Newton's method is by induction. Corollary 4.1 Let f be twice continuously dierentiable in an open set D where the second partial derivatives are Lipschitz ... |

501 |
Constrained Optimization and Lagrange multiplier method
- Bertsekas
- 1997
(Show Context)
Citation Context ...n conrmed to satisfy the traditional criteria required for global convergence. We point out that the conservative two-step line-search algorithm is not new and can be found in the books by Bertsekas [=-=1]-=-, Section 1.3.1, and Luenberger [19], Section 7.10, where the second step is called a spacer step. A description of the greedy and conservative two-step line-search algorithms can be found in [11]. In... |

335 | SNOPT: An SQP algorithm for large-scale constrained optimization
- Gill, Murray, et al.
(Show Context)
Citation Context ...+ c i (x k ) ; i = 1; : : : ; m : (27) It is important to remark that these updates require no further function or gradient evaluations. They have also been considered in the codes NPSOL and SNOPT [1=-=5]-=-, [16] to update slack variables after the application of a line search to the augmented Lagrangian merit function and prior to the solution of the next quadratic programming problem. Other ways of de... |

210 |
Computing a trust region step
- Moré, Sorensen
- 1983
(Show Context)
Citation Context ...n of the scalar problem in the unknown minimize m k (y k + s) subject to ksk k ; s = rf(y k ) ; 2 IR : There is a variety of algorithms that compute steps satisfying this condition (see [3], [22], [23], [25], and [26]). TWO-STEP ALGORITHMS FOR NONLINEAR OPTIMIZATION 6 Proposition 3.1 If s k satises a fraction of Cauchy decrease then: f(y k ) m k (y k + s k ) 2 krf(y k )k min k ; kr... |

127 |
The conjugate gradient method and trust regions in large scale optimization
- Steihaug
- 1983
(Show Context)
Citation Context ...m in the unknown minimize m k (y k + s) subject to ksk k ; s = rf(y k ) ; 2 IR : There is a variety of algorithms that compute steps satisfying this condition (see [3], [22], [23], [25], and [26]). TWO-STEP ALGORITHMS FOR NONLINEAR OPTIMIZATION 6 Proposition 3.1 If s k satises a fraction of Cauchy decrease then: f(y k ) m k (y k + s k ) 2 krf(y k )k min k ; krf(y k )k (7) where... |

123 |
CUTEr, a Constrained and Unconstrained Test Environment, revisited. http://cuter.rl.ac.uk/cuter-www
- Toint
(Show Context)
Citation Context ...on 2 but with no update of the variable z for minimax problems, i.e., zsxed in (34)-(36). We compared the numerical performance of these three versions on a set of problems 1 from the CUTE collection =-=[2]-=-. This set of problems is listed in Table 1, and in the case of minimax formulations in Table 2, where we mention the number of variables (including slacks and, where applicable, the minimax variable ... |

62 |
Global convergence of a class of trust region algorithms for optimization with simple bounds
- Toint
- 1988
(Show Context)
Citation Context ...-region algorithm. We are motivated by a situation where it is desirable to update slack variables and variables introduced to solve minimax problems, at every iteration of the trust-region algorithm =-=[7-=-] implemented in LANCELOT [9]. See Section 5 for more details on practical applications. The two-step trust-region algorithm is quite easy to describe. Suppose that after computing a step s k based o... |

53 |
User’S Guide For Npsol 5.0: A Fortran Package For Nonlinear Programming
- Gill, Murray, et al.
- 1998
(Show Context)
Citation Context ...(x k ) ; i = 1; : : : ; m : (27) It is important to remark that these updates require no further function or gradient evaluations. They have also been considered in the codes NPSOL and SNOPT [15], [1=-=6]-=- to update slack variables after the application of a line search to the augmented Lagrangian merit function and prior to the solution of the next quadratic programming problem. Other ways of dealing ... |

32 |
Minimization of a large-scale quadratic functionsubject to a spherical constraint
- Sorensen
- 1997
(Show Context)
Citation Context ...lar problem in the unknown minimize m k (y k + s) subject to ksk k ; s = rf(y k ) ; 2 IR : There is a variety of algorithms that compute steps satisfying this condition (see [3], [22], [23], [25], and [26]). TWO-STEP ALGORITHMS FOR NONLINEAR OPTIMIZATION 6 Proposition 3.1 If s k satises a fraction of Cauchy decrease then: f(y k ) m k (y k + s k ) 2 krf(y k )k min k ; krf(y k )k ... |

29 |
Approximate solution of the trust region problem by minimization over two-dimensional subspaces
- Byrd, Schnabel, et al.
- 1988
(Show Context)
Citation Context ...lution of the scalar problem in the unknown minimize m k (y k + s) subject to ksk k ; s = rf(y k ) ; 2 IR : There is a variety of algorithms that compute steps satisfying this condition (see [3], [22], [23], [25], and [26]). TWO-STEP ALGORITHMS FOR NONLINEAR OPTIMIZATION 6 Proposition 3.1 If s k satises a fraction of Cauchy decrease then: f(y k ) m k (y k + s k ) 2 krf(y k )k min ... |

26 | Gradient-based optimization of custom circuits using a static-timing formulation
- Conn, Elfadel, et al.
- 1999
(Show Context)
Citation Context ...lation. There are several nonlinear optimization problems in which some subset of the problem variables occur linearly, for example, arrival times in static-timing-based circuit optimization problems =-=[-=-6]. Such problems can also benet from two-step updating. 6 Numerical tests 6.1 Analytic problems We modied LANCELOT (Release A) [9] to include the slack variable update (27) and the slack and minimax ... |

26 |
e, Recent developments in algorithms and software for trust region methods
- Mor'
- 1983
(Show Context)
Citation Context ...ng-based circuit optimization problems. Finally, some conclusions are drawn in Section 7. 2 Two-step trust-region algorithms Wesrst consider the trust-region framework presented in the paper by More [=-=20]-=- for unconstrained minimization. The (classical) trust-region algorithm builds a quadratic model of the form m k (y k + s) = f(y k ) +rf(y k ) > s + 1 2 s > H k s at the current point y k , where H k ... |

20 |
A Fortran Package for Large-Scale Nonlinear Optimization (Release A
- LANCELOT
- 1992
(Show Context)
Citation Context ...tivated by a situation where it is desirable to update slack variables and variables introduced to solve minimax problems, at every iteration of the trust-region algorithm [7] implemented in LANCELOT =-=[9-=-]. See Section 5 for more details on practical applications. The two-step trust-region algorithm is quite easy to describe. Suppose that after computing a step s k based on the trust-region subproble... |

20 |
Piecewise approximate circuit simulation
- Visweswariah, Rohrer
- 1991
(Show Context)
Citation Context ...es transistor and wire sizes of digital integrated circuits to meet delay, power, and area goals. It is based on fast circuit simulation and time-domain sensitivity computation in SPECS (see [13] and =-=[28]-=-). To optimize multiple path delays through a high-performance circuit, the tuning is often formulated as a minimax problem or a minimization problem with nonlinear inequality constraints. We remark t... |

16 |
Sensitivity computation in piecewise approximate circuit simulation
- Feldmann, Nguyen, et al.
- 1991
(Show Context)
Citation Context ...e optimizes transistor and wire sizes of digital integrated circuits to meet delay, power, and area goals. It is based on fast circuit simulation and time-domain sensitivity computation in SPECS (see =-=[13]-=- and [28]). To optimize multiple path delays through a high-performance circuit, the tuning is often formulated as a minimax problem or a minimization problem with nonlinear inequality constraints. We... |

14 |
Jiffytune: circuit optimization using time-domain sensitivities,” Computer-Aided Design of Integrated Circuits and Systems
- Conn, Coulman, et al.
- 1998
(Show Context)
Citation Context ...ned to terminate before many \asymptotic" iterations are taken. The algorithms described in this paper have been used in a dynamic-simulation-based circuit optimization tool called JiyTune (see [=-=4], [5-=-], and [10]). JiyTune optimizes transistor and wire sizes of digital integrated circuits to meet delay, power, and area goals. It is based on fast circuit simulation and time-domain sensitivity comput... |

14 |
The dierentiation of pseudo-inverses and nonlinear least squares problems whose variables separate
- Golub, Pereyra
- 1973
(Show Context)
Citation Context ...e approach, when feasible, is to reformulate the original problem by eliminating a subset of variables and then to apply the algorithms in the remaining variables (see, for example, Golub and Pereyra =-=[17]-=-). This paper deals with two-step algorithms where the second step is required to yield a decrease in the value of the objective function. The analysis given here covers the global convergence of two-... |

14 |
Sequential estimation techniques for Quasi-Newton algorithms
- Thomas
- 1975
(Show Context)
Citation Context ...m k !+1 krf(y k )k = 0 : (14) So, if the sequence fy k g is bounded, every limit point y satises rf(y ) = 0. Proof: The proof is similar to the proof given in [20], Theorem 4.14. See also Thomas [27=-=-=-]. We show the result by contradiction. Assume therefore that there exists an 1 2 (0; 1) and a subsequence indexed by fm i g of successful iterates such that, for all m i in this subsequence, krf(ym ... |

11 |
A new algorithm for unconstrained optimization. In: Nonlinear Programming (Proc
- Powell
- 1970
(Show Context)
Citation Context ...he scalar problem in the unknown minimize m k (y k + s) subject to ksk k ; s = rf(y k ) ; 2 IR : There is a variety of algorithms that compute steps satisfying this condition (see [3], [22], [23], [25], and [26]). TWO-STEP ALGORITHMS FOR NONLINEAR OPTIMIZATION 6 Proposition 3.1 If s k satises a fraction of Cauchy decrease then: f(y k ) m k (y k + s k ) 2 krf(y k )k min k ; krf(y k ... |

10 | Optimization of custom mos circuits by transistor sizing
- Conn, Coulman, et al.
- 1996
(Show Context)
Citation Context ...designed to terminate before many \asymptotic" iterations are taken. The algorithms described in this paper have been used in a dynamic-simulation-based circuit optimization tool called JiyTune (=-=see [4-=-], [5], and [10]). JiyTune optimizes transistor and wire sizes of digital integrated circuits to meet delay, power, and area goals. It is based on fast circuit simulation and time-domain sensitivity c... |

6 | Circuit optimization via adjoint Lagrangians
- Conn, Haring, et al.
- 1997
(Show Context)
Citation Context ...rminate before many \asymptotic" iterations are taken. The algorithms described in this paper have been used in a dynamic-simulation-based circuit optimization tool called JiyTune (see [4], [5], =-=and [10-=-]). JiyTune optimizes transistor and wire sizes of digital integrated circuits to meet delay, power, and area goals. It is based on fast circuit simulation and time-domain sensitivity computation in S... |

2 |
of the trust region problem
- Generalizations
- 1993
(Show Context)
Citation Context ...is basically the same as the proof of Theorem 4.7 in [22]. 2 To obtain stronger global convergence results to second-order points, for instance the results in Theorems 4.11 and 4.13 in [22] (see also =-=[21-=-], Theorem 4.17, c and d), other conditions are required like k^s k k being of O( k ). The next results show that the second step can preserve the nice local properties of the behavior of the trust ra... |

1 |
On solving three classes of nonlinear programming problems via simple dierentiable penalty functions
- Gould
- 1988
(Show Context)
Citation Context ... the augmented Lagrangian merit function and prior to the solution of the next quadratic programming problem. Other ways of dealing with slack variables have been studied in the literature (see Gould =-=[18]-=- and the references therein). For the study of the impact of the slack variable update on the global convergence of the trustregion algorithm, the step in these variables is required only to decrease ... |