## Nonmonotone spectral projected gradient methods on convex sets (2000)

### Cached

### Download Links

Venue: | SIAM Journal on Optimization |

Citations: | 135 - 25 self |

### BibTeX

@ARTICLE{Birgin00nonmonotonespectral,

author = {Ernesto G. Birgin and Jos É Mario Martínez and Marcos Raydan},

title = {Nonmonotone spectral projected gradient methods on convex sets},

journal = {SIAM Journal on Optimization},

year = {2000},

pages = {1196--1211}

}

### Years of Citing Articles

### OpenURL

### Abstract

Abstract. Nonmonotone projected gradient techniques are considered for the minimization of differentiable functions on closed convex sets. The classical projected gradient schemes are extended to include a nonmonotone steplength strategy that is based on the Grippo–Lampariello–Lucidi nonmonotone line search. In particular, the nonmonotone strategy is combined with the spectral gradient choice of steplength to accelerate the convergence process. In addition to the classical projected gradient nonlinear path, the feasible spectral projected gradient is used as a search direction to avoid additional trial projections during the one-dimensional search process. Convergence properties and extensive numerical results are presented.

### Citations

161 |
Two point step size gradient method
- Barzilai, Borwein
- 1988
(Show Context)
Citation Context ... the nonmonotone line search schemes developed by Grippo, Lampariello, and Lucidi [17] for Newton’s method. Second, we propose to associate the spectral steplength, introduced by Barzilai and Borwein =-=[1]-=- and analyzed by Raydan [26]. This choice of steplength requires little computational work and greatly speeds up the convergence of gradient methods. In fact, while the spectral gradient method appear... |

148 |
A Nonmonotone Line Search Technique for Newton's Method
- Grippo, Lampariello, et al.
- 1986
(Show Context)
Citation Context ...oped ingredients in optimization. First we extend the typical globalization strategies associated with these methods to the nonmonotone line search schemes developed by Grippo, Lampariello and Lucidi =-=[17]-=- for Newton’s method. Second, we propose to associate the spectral steplength, introduced by Barzilai and Borwein [1] and analyzed by Raydan [26]. This choice of steplength requires little computation... |

96 |
LANCELOT: A Fortran Package for LargeScale Nonlinear Optimization (Release A
- Toint
- 1991
(Show Context)
Citation Context ... 1 other academic 2 other modeling 3 other real application 4 sum of squares academic 5 sum of squares modeling 6 quadratic academic 7 quadratic modeling 8 quadratic real application package LANCELOT =-=[9]-=- using all the bound constrained problems with more than 50 variables from the CUTE [10] collection. Only problem GRIDGENA was excluded from our tables because it gives an “exception error” when evalu... |

82 |
The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem
- Raydan
- 1997
(Show Context)
Citation Context ...e previous step. See Glunt et al. [15] for a relationship with the shifted power method to approximate eigenvalues and eigenvectors, and also for an interesting chemistry application. See also Raydan =-=[27]-=- for a combination of the spectral choice of steplength with nonmonotone line search techniques to solve unconstrained minimization problems. An successful application of this technique can be found i... |

74 |
Projected gradient methods for linearly constrained problems
- Calamai, Moré
- 1987
(Show Context)
Citation Context ...uous projected path that will be properly defined in section 2. The convergence properties of the projected gradient method for different choices of stepsize have been extensively studied. See, e.g., =-=[2, 3, 7, 11, 16, 19, 22, 30]-=-. For an interesting review of the different convergence results that have been obtained under different assumptions, see Calamai and Moré [7]. For a complete survey see Dunn [12]. In section 2 of thi... |

72 |
On the Goldstein-Levitin-Polyak gradient projection method
- Bertsekas
- 1976
(Show Context)
Citation Context ... nonmonotone line search to the projected gradient case in order to speed up the convergence of the projected gradient method. In particular, in this work we extend the practical version of Bertsekas =-=[2]-=- that enforces an Armijo-type condition along the curvilinear projection path. This practical version is based on the original version proposed by Goldstein [16] and Levitin and Polyak [19]. We also a... |

62 |
Global convergence of a class of trust region algorithms for optimization with simple bounds
- Toint
- 1988
(Show Context)
Citation Context ... infinite bounds. In fact, good algorithms for box constrained minimization are the essential tool for the development of efficient augmented Lagrangian methods for general nonlinear programming (see =-=[8, 10, 13]-=-). With this in mind, we implemented the spectral projected gradient algorithms for the case in which Ω is described by bounds on the variables. In order to assess the reliability of SPG algorithms, w... |

60 | A semidefinite framework for trust region subproblems with applications to large scale minimization
- Rendl, Wolkowicz
- 1997
(Show Context)
Citation Context ...m nonglobal stationary points of this problem have been found and, so, it becomes increasingly important to obtain fast algorithms for finding critical points especially in the large-scale case. (See =-=[28, 29, 31]-=-.) Perhaps the most important characteristic of SPG algorithms is that they are extremely simple to code, at a point that anyone can write her/his own code using any scientific language in a couple of... |

54 |
Molecular conformations from distance matrices
- Glunt, Hayden, et al.
- 1993
(Show Context)
Citation Context ... stepsize that comes from the minimization of the function along the gradient of the current iteration, the one that comes from the one-dimensional minimization at the previous step. See Glunt et al. =-=[15]-=- for a relationship with the shifted power method to approximate eigenvalues and eigenvectors, and also for an interesting chemistry application. See also Raydan [27] for a combination of the spectral... |

53 |
On the Barzilai and Borwein choice of steplength for the gradient method
- Raydan
- 1993
(Show Context)
Citation Context ...ch schemes developed by Grippo, Lampariello and Lucidi [17] for Newton’s method. Second, we propose to associate the spectral steplength, introduced by Barzilai and Borwein [1] and analyzed by Raydan =-=[26]-=-. This choice of steplength requires little computational work and greatly speeds up the convergence of gradient methods. In fact, while the spectral gradient method appears to be a generalized steepe... |

53 |
A globally convergent augmented Lagrangian algorithm for optimization with general constraints and simple bounds
- Toint
- 1991
(Show Context)
Citation Context ... infinite bounds. In fact, good algorithms for box constrained minimization are the essential tool for the development of efficient augmented Lagrangian methods for general nonlinear programming (see =-=[8, 10, 13]-=-). With this in mind, we implemented the spectral projected gradient algorithms for the case in which Ω is described by bounds on the variables. In order to assess the reliability of SPG algorithms, w... |

49 | A new matrix-free algorithm for the large-scale trust-region subproblem
- Rojas, Santos, et al.
(Show Context)
Citation Context ...m nonglobal stationary points of this problem have been found and, so, it becomes increasingly important to obtain fast algorithms for finding critical points especially in the large-scale case. (See =-=[28, 29, 31]-=-.) Perhaps the most important characteristic of SPG algorithms is that they are extremely simple to code, at a point that anyone can write her/his own code using any scientific language in a couple of... |

47 |
A new trust region algorithm for bound constrained minimization
- Friedlander, Martínez, et al.
- 1994
(Show Context)
Citation Context ... infinite bounds. In fact, good algorithms for box constrained minimization are the essential tool for the development of efficient augmented Lagrangian methods for general nonlinear programming (see =-=[8, 10, 13]-=-). With this in mind, we implemented the spectral projected gradient algorithms for the case in which Ω is described by bounds on the variables. In order to assess the reliability of SPG algorithms, w... |

46 |
Convex programming in Hilbert space
- GOLDSTEIN
- 1964
(Show Context)
Citation Context ...tend the practical version of Bertsekas [2] that enforces an Armijo-type condition along the curvilinear projection path. This practical version is based on the original version proposed by Goldstein =-=[16]-=- and Levitin and Polyak [19]. We also apply the new ingredients to the feasible continuous projected path that will be properly defined in section 2. The convergence properties of the projected gradie... |

35 |
Global and asymptotic convergence rate estimates for a class of projected gradient processes
- Dunn
- 1981
(Show Context)
Citation Context ... continuous projected path that will be properly defined in Section 2. The convergence properties of the projected gradient method for different choices of stepsize have been extensively studied. See =-=[2, 3, 7, 11, 16, 19, 22, 30]-=-, and other authors. For an interesting review of the different convergence results that have been obtained under different assumptions, see Calamai and Moré [7]. For a complete survey see Dunn [12]. ... |

35 | Inexact spectral projected gradient methods on convex sets
- Birgin, Martínez, et al.
(Show Context)
Citation Context ...t, i.e., 〈g(¯x), x − ¯x〉 ≥ 0 for all x ∈ Ω. Theorem 2.1 Algorithm SPG2 is well defined, and any accumulation point of the sequence {xk} that it generates is a constrained stationary point. Proof. See =-=[7]-=-. ✷ The proof of the following theorem relies on Proposition 2.3.3 in Bertsekas [3], which is related to the Armijo condition along the projection arc. This proposition was originally shown in [15]. F... |

32 |
Minimization of a large-scale quadratic functionsubject to a spherical constraint
- Sorensen
- 1997
(Show Context)
Citation Context ...m nonglobal stationary points of this problem have been found, and so it becomes increasingly important to obtain fast algorithms for finding critical points, especially in the large-scale case. (See =-=[28, 29, 31]-=-.) Perhaps the most important characteristic of SPG algorithms is that they are extremely simple to code, to the point that anyone can write her or his own code using any scientific language in a coup... |

26 | Estimation of the optical constants and the thickness of thin films using unconstrained optimization
- BIRGIN, CHAMBOULEYRON, et al.
- 1999
(Show Context)
Citation Context ...for a combination of the spectral choice of steplength with nonmonotone line search techniques to solve unconstrained minimization problems. A successful application of this technique can be found in =-=[5]-=-. Therefore, it is natural and rather easy to transport the spectral gradient idea with a nonmonotone line search to the projected gradient case in order to speed up the convergence of the projected g... |

25 |
Constrained minimization problems
- LEVITIN, POLJAK
- 1966
(Show Context)
Citation Context ...f Bertsekas [2] that enforces an Armijo-type condition along the curvilinear projection path. This practical version is based on the original version proposed by Goldstein [16] and Levitin and Polyak =-=[19]-=-. We also apply the new ingredients to the feasible continuous projected path that will be properly defined in section 2. The convergence properties of the projected gradient method for different choi... |

21 | Restricted optimization: a clue to a fast and accurate implementation of the Common Reflection Surface method - Birgin, Biloti, et al. - 1999 |

18 |
Preconditioned spectral gradient method
- Luengo, Raydan, et al.
- 2002
(Show Context)
Citation Context ...mal value is ≈ 0.1 %. Slow convergence of SPG methods when the Hessian at the local minimizer is very ill-conditioned is expected and pre-conditioning schemes tend to alleviate this inconvenient. See =-=[21]-=-. In the remaining 40 problems, LANCELOT, SPG1 and SPG2 found the same solutions. In terms of computer time, SPG1 was faster than LANCELOT in 29 problems (72.5 %) and SPG2 outperformed LANCELOT also i... |

18 |
Minimization of a large scale quadratic function subject to an spherical constraint
- SORENSEN
- 1997
(Show Context)
Citation Context ...m nonglobal stationary points of this problem have been found and, so, it becomes increasingly important to obtain fast algorithms for finding critical points especially in the large-scale case. (See =-=[28, 29, 31]-=-.) Perhaps the most important characteristic of SPG algorithms is that they are extremely simple to code, at a point that anyone can write her/his own code using any scientific language in a couple of... |

16 |
The gradient projection method under mild differentiability conditions
- McCormick, Tapia
- 1972
(Show Context)
Citation Context ...uous projected path that will be properly defined in section 2. The convergence properties of the projected gradient method for different choices of stepsize have been extensively studied. See, e.g., =-=[2, 3, 7, 11, 16, 19, 22, 30]-=-. For an interesting review of the different convergence results that have been obtained under different assumptions, see Calamai and Moré [7]. For a complete survey see Dunn [12]. In section 2 of thi... |

15 | A trust-region strategy for minimization on arbitrary domains
- Martínez, Santos
- 1995
(Show Context)
Citation Context ...ible set when this is simple enough, a fact that is fully exploited in SPG1 and SPG2. Boxes are not the only type of sets on which it is trivial to project. The normconstrained regularization problem =-=[18, 23, 24, 32]-=-, defined by Minimize f(x) subject to x T Ax ≤ r (15) where A is symmetric positive definite can be reduced to ball constrained minimization by a change of variables and, in this case, projections can... |

11 | Automatic differentiation and spectral projected gradient methods for optimal control problems - Birgin, Evtushenko - 1998 |

7 |
Toint [1991], A globally convergent augmented Lagrangian algorithm for optimization with general constraints and simple bounds
- unknown authors
(Show Context)
Citation Context |

7 |
Convergence of a gradient projection method
- Gafni, Bertsekas
- 1982
(Show Context)
Citation Context ...for all x ∈ Ω. The proof of our first theorem relies on Proposition 2.3.3 in Bertsekas [3], which is related to the Armijo condition along the projection arc. This proposition was originally shown in =-=[14]-=-. For completeness we include in the next lemma some technical results from [3] that will be used in our proof. Lemma 2.2 (i) For all x ∈ Ω and z ∈ IR n , the function h : [0, ∞) → IR given by h(s) = ... |

7 |
Mesh independence for nonlinear least squares problems with norm constraints
- Heinkenschloss
- 1993
(Show Context)
Citation Context ...ible set when this is simple enough, a fact that is fully exploited in SPG1 and SPG2. Boxes are not the only type of sets on which it is trivial to project. The normconstrained regularization problem =-=[18, 23, 24, 32]-=-, defined by Minimize f(x) subject to x T Ax ≤ r (15) where A is symmetric positive definite can be reduced to ball constrained minimization by a change of variables and, in this case, projections can... |

7 | On some properties of quadratic programs with a convex quadratic constraint
- LUCIDI, PALAGI, et al.
- 1998
(Show Context)
Citation Context ...minimization by a change of variables and, in this case, projections can be trivially computed. A particular case of (15) is the classical trust-region subproblem, where f is quadratic. Recently (see =-=[20, 25]-=-) procedures for escaping from nonglobal stationary points of this problem have been found and, so, it becomes increasingly important to obtain fast algorithms for finding critical points especially i... |

7 |
Family of projected descent methods for optimization problems with simple bounds
- Schwartz, Polak
- 1997
(Show Context)
Citation Context ... continuous projected path that will be properly defined in Section 2. The convergence properties of the projected gradient method for different choices of stepsize have been extensively studied. See =-=[2, 3, 7, 11, 16, 19, 22, 30]-=-, and other authors. For an interesting review of the different convergence results that have been obtained under different assumptions, see Calamai and Moré [7]. For a complete survey see Dunn [12]. ... |

6 |
A constrained least squares regularization method for nonlinear ill--posed problems
- Vogel
- 1990
(Show Context)
Citation Context ...ible set when this is simple enough, a fact that is fully exploited in SPG1 and SPG2. Boxes are not the only type of sets on which it is trivial to project. The normconstrained regularization problem =-=[18, 23, 24, 32]-=-, defined by Minimize f(x) subject to x T Ax ≤ r (15) where A is symmetric positive definite can be reduced to ball constrained minimization by a change of variables and, in this case, projections can... |

5 |
Toint [1988], Global convergence of a class of trust region algorithms for optimization with simple bounds
- unknown authors
- 1989
(Show Context)
Citation Context |

5 | 1997], Convergence results on an algorithm for norm constrained regularization and related problems
- Martínez, Santos
(Show Context)
Citation Context |

4 |
Borwein [1988], Two point step size gradient methods
- Barzilai, M
(Show Context)
Citation Context ...o the nonmonotone line search schemes developed by Grippo, Lampariello and Lucidi [17] for Newton’s method. Second, we propose to associate the spectral steplength, introduced by Barzilai and Borwein =-=[1]-=- and analyzed by Raydan [26]. This choice of steplength requires little computational work and greatly speeds up the convergence of gradient methods. In fact, while the spectral gradient method appear... |

4 |
Goldstein [1964], Convex Programming in Hilbert Space
- A
(Show Context)
Citation Context ...d the practical version of Bertsekas [2] that enforces an Armijo type of condition along the curvilinear projection path. This practical version is based on the original version proposed by Goldstein =-=[16]-=- and Levitin and Polyak [19]. We also apply the new ingredients to the feasible continuous projected path that will be properly defined in Section 2. The convergence properties of the projected gradie... |

3 |
Convergence of a gradient project method, Report LIDS-P-1201
- Gafni, Bertsekas
- 1982
(Show Context)
Citation Context ...for all x ∈ Ω. The proof of our first theorem relies on Proposition 2.3.3 in Bertsekas [3], which is related to the Armijo condition along the projection arc. This proposition was originally shown in =-=[14]-=-. For completeness we include in the next lemma some technical results from [3] that will be used in our proof. Lemma 2.2. (i) For all x ∈ Ω and z ∈ R n , the function h :[0, ∞) → R given by ‖P (x + s... |

2 |
Bertsekas [1976], On the Goldstein-Levitin-Polyak gradient projection method
- P
(Show Context)
Citation Context ... nonmonotone line search to the projected gradient case in order to speed up the convergence of the projected gradient method. In particular, in this work we extend the practical version of Bertsekas =-=[2]-=- that enforces an Armijo type of condition along the curvilinear projection path. This practical version is based on the original version proposed by Goldstein [16] and Levitin and Polyak [19]. We als... |

2 |
Martínez [1999], Estimation of the optical constants and the thickness of thin films using unconstrained optimization
- Birgin, Chambouleyron, et al.
(Show Context)
Citation Context ...or a combination of the spectral choice of steplength with nonmonotone line search techniques to solve unconstrained minimization problems. An successful application of this technique can be found in =-=[5]-=-. Therefore, it is natural and rather easy to transport the spectral gradient idea with a nonmonotone line search to the projected gradient case in order to speed up the convergence of the projected g... |

2 |
1994], Gradient-Related Constrained Minimization Algorithms in Function Spaces
- Dunn
(Show Context)
Citation Context ...2, 30], and other authors. For an interesting review of the different convergence results that have been obtained under different assumptions, see Calamai and Moré [7]. For a complete survey see Dunn =-=[12]-=-. In Section 2 of this paper we define the spectral projected gradient algorithms and 2swe prove global convergence results. In Section 3 we present numerical experiments. This set of experiments show... |

2 |
Hoai An [1998], D.C. Optimization algorithm for solving the trust region problem
- Dinh, T
(Show Context)
Citation Context ...minimization by a change of variables and, in this case, projections can be trivially computed. A particular case of (15) is the classical trust-region subproblem, where f is quadratic. Recently (see =-=[20, 25]-=-) procedures for escaping from nonglobal stationary points of this problem have been found and, so, it becomes increasingly important to obtain fast algorithms for finding critical points especially i... |

1 | Evtushenko [1998], Automatic differentiation and spectral projected gradient methods for optimal control problems - Birgin, G |

1 |
Moré [1987], Projected gradient methods for linearly constrained problems
- Calamai, J
(Show Context)
Citation Context ... continuous projected path that will be properly defined in Section 2. The convergence properties of the projected gradient method for different choices of stepsize have been extensively studied. See =-=[2, 3, 7, 11, 16, 19, 22, 30]-=-, and other authors. For an interesting review of the different convergence results that have been obtained under different assumptions, see Calamai and Moré [7]. For a complete survey see Dunn [12]. ... |

1 |
Tapia [1972], The gradient projection method under mild differentiability conditions
- McCormick, A
(Show Context)
Citation Context |

1 |
optimization algorithm for solving the trust-region subproblem
- Tao, An, et al.
- 1998
(Show Context)
Citation Context ...minimization by a change of variables and, in this case, projections can be trivially computed. A particular case of (15) is the classical trust-region subproblem, where f is quadratic. Recently (see =-=[20, 25]-=-) procedures for escaping from nonglobal stationary points of this problem have been found, and so it becomes increasingly important to obtain fast algorithms for finding critical points, especially i... |