## Primal-dual projected gradient algorithms for extended linearquadratic programming

Venue: | SIAM J. Optimization |

Citations: | 17 - 2 self |

### BibTeX

@ARTICLE{Zhu_primal-dualprojected,

author = {Ciyou Zhu and R. T. Rockafellar},

title = {Primal-dual projected gradient algorithms for extended linearquadratic programming},

journal = {SIAM J. Optimization},

year = {}

}

### Years of Citing Articles

### OpenURL

### Abstract

Abstract. Many large-scale problems in dynamic and stochastic optimization can be modeled with extended linear-quadratic programming, which admits penalty terms and treats them through duality. In general the objective functions in such problems are only piecewise smooth and must be minimized or maximized relative to polyhedral sets of high dimensionality. This paper proposes a new class of numerical methods for “fully quadratic ” problems within this framework, which exhibit second-order nonsmoothness. These methods, combining the idea of finite-envelope representation with that of modified gradient projection, work with local structure in the primal and dual problems simultaneously, feeding information back and forth to trigger advantageous restarts. Versions resembling steepest descent methods and conjugate gradient methods are presented. When a positive threshold of ε-optimality is specified, both methods converge in a finite number of iterations. With threshold 0, it is shown under mild assumptions that the steepest descent version converges linearly, while the conjugate gradient version still has a finite termination property. The algorithms are designed to exploit features of primal and dual decomposability of the Lagrangian, which are typically available in a large-scale setting, and they are open to considerable parallelization. Key words. Extended linear-quadratic programming, large-scale numerical optimization, finite-envelope representation, gradient projection, primal-dual methods, steepest descent methods, conjugate gradient methods. AMS(MOS) subject classifications. 65K05, 65K10, 90C20 1. Introduction. A

### Citations

3777 | Convex Analysis
- Rockafellar
- 1970
(Show Context)
Citation Context ...n made in Rockafellar [6]. Other ideas, which involve splitting algorithms, have been explored by Tseng [15], [16]. Here we aim at adapting classical descent algorithms with help from convex analysis =-=[17]-=-. In this paper we make the blanket assumption of double decomposability, taking it as license also for exact line searchability [6]: the supposition that it is possible to minimize f(u) over any line... |

871 | Nonlinear Programming: Theory and Algorithms - Bazaraa, Sherali, et al. - 1993 |

309 |
Monotone operators and the proximal point algorithm
- Rockafellar
- 1976
(Show Context)
Citation Context ...otential in mathematical modeling (cf. [2], [4]) and because strong convexity-concavity of the Lagrangian can be created, if need be, through some outer implementation of the proximal point algorithm =-=[18]-=-, [19], as carried out in [1] and [8]. The questions concerning such an outer algorithm are best handled elsewhere, since they have a different character and relate to a host of primal-dual procedures... |

184 |
Interior path-following primal-dual algorithms
- Monteiro, Adler
- 1989
(Show Context)
Citation Context ...s [10], where U = lR n +, V = lR m + , and Q is not necessarily the zero matrix, are surveyed by Lin and Pang [11]. Other efforts in recent times have been made by Ye and Tse [12], Monteiro and Adler =-=[13]-=-, Goldfarb and Liu [14]. None of these approaches is consonant with the large-scale applications that attract our interest, because the structure in such applications is not well served by the wholesa... |

153 |
Nonlinear programming, Analysis and methods
- Avriel
- 1976
(Show Context)
Citation Context ...problem. Note that all the eigenvalues of the Hessian matrix Aff are in the interval [1, 1 + ‖ ˜ Rff ‖2 ], where ‖ ˜ Rff ‖ 2 ≤ ‖ ˜ R‖ 2 = γ 2 . Hence by using the expression of ˜ f in (4.11), we have =-=[22]-=- (4.23) ˜f(ˆũ ν+1 0 ) − ˜ f(¯ũ) ˜f(ũ ν 0 ) − ˜ f(¯ũ) ≤ ( ‖ ˜ Rff ‖2 ‖ ˜ Rff ‖2 + 2 ) 2 ≤ ( 1 − 1 1 + 1 2‖ ˜ R‖2 ) 2 , which yields (4.20) since ˜ f(ũ ν+1 0 ) ≤ f(ˆũ ν+1 0 ) in the algorithm. In Case 2... |

146 |
Augmented Lagrangians and applications of the proximal point algorithm in convex programming,Mathematics of Operations Research 1
- Rockafellar
- 1976
(Show Context)
Citation Context ...al in mathematical modeling (cf. [2], [4]) and because strong convexity-concavity of the Lagrangian can be created, if need be, through some outer implementation of the proximal point algorithm [18], =-=[19]-=-, as carried out in [1] and [8]. The questions concerning such an outer algorithm are best handled elsewhere, since they have a different character and relate to a host of primal-dual procedures in ex... |

72 |
Applications of a splitting algorithm to decomposition in convex programming and variational inequalities
- Tseng
- 1991
(Show Context)
Citation Context ...make use of such information in the design of numerical methods. Some proposals have already been made in Rockafellar [6]. Other ideas, which involve splitting algorithms, have been explored by Tseng =-=[15]-=-, [16]. Here we aim at adapting classical descent algorithms with help from convex analysis [17]. In this paper we make the blanket assumption of double decomposability, taking it as license also for ... |

55 |
On the identification of active constraints
- Burke, Moré
- 1988
(Show Context)
Citation Context ...uch as to insert at the beginning of each primal cycle a line search in the direction of the projection of −∇f(uν 0 ) on the tangent cone to U at uν 0 , and similarly in the dual. (See Burke and Moré =-=[23]-=-.) But even without such remedies, we often find in our test problems that the critical faces are identified in the tail of iteration, and that restarts do occur in most cases, after which the iterati... |

35 | Wets, ”Generalized linear-quadratic problems of deterministic and stochastic optimal control in discrete time - Rockafellar, J-B - 1990 |

30 |
Linear-quadratic programming and optimal control
- Rockafellar
- 1987
(Show Context)
Citation Context ...n [1], [2], first used the concept in two-stage stochastic programming, where the primal dimension is low but the dual dimension is high. It was developed further in its own right in Rockafellar [3], =-=[4]-=-, and carried in the latter paper to the context of continuous-time optimal control. Discrete-time problems of optimal control, both deterministic and stochastic (i.e., multistage stochastic programmi... |

26 |
Wets, ”A Lagrangian finite generation technique for solving linear-quadratic problems in stochastic programming,” Mathematical Programming Study 28
- Rockafellar, J-B
- 1986
(Show Context)
Citation Context ...f optimization than conventional quadratic programming and seems especially suited to large-scale applications, in particular because of way penalty terms can be incorporated. Rockafellar and Wets in =-=[1]-=-, [2], first used the concept in two-stage stochastic programming, where the primal dimension is low but the dual dimension is high. It was developed further in its own right in Rockafellar [3], [4], ... |

26 |
User’s guide for lssol (version 1.0): A fortran package for constrained linear least-squares and convex quadratic programming
- Gill, Hammarling, et al.
- 1986
(Show Context)
Citation Context ...cision on some medium to large-sized problems. For comparisons we have used the Basic Finite Envelope Method (BFEM) of [6] and the Stanford LSSOL code of Gill, Hammarling, Murray, Saunders and Wright =-=[24]-=- for quadratic programming. To enhance the performance of LSSOL in this situation, we tailored its Cholesky factorization subroutine to take advantage of the special structure of the P and Q matrices ... |

21 |
An O(n3L) primal interior point algorithm for convex quadratic programming
- Goldfarb, Liu
- 1991
(Show Context)
Citation Context ...+, V = lR m + , and Q is not necessarily the zero matrix, are surveyed by Lin and Pang [11]. Other efforts in recent times have been made by Ye and Tse [12], Monteiro and Adler [13], Goldfarb and Liu =-=[14]-=-. None of these approaches is consonant with the large-scale applications that attract our interest, because the structure in such applications is not well served by the wholesale reformulations that ... |

19 |
Iterative methods for large convex quadratic programs: A survey
- Lin, Pang
- 1987
(Show Context)
Citation Context ...uadratic programming, and the somewhat more general area of linear complementarity problems [10], where U = lR n +, V = lR m + , and Q is not necessarily the zero matrix, are surveyed by Lin and Pang =-=[11]-=-. Other efforts in recent times have been made by Ye and Tse [12], Monteiro and Adler [13], Goldfarb and Liu [14]. None of these approaches is consonant with the large-scale applications that attract ... |

17 |
Wets, ”Linear quadratic problems with stochastic penalties: the finite generation algorithm,” in:Numerical Techniques for Stochastic Optimization
- Rockafellar, J-B
- 1987
(Show Context)
Citation Context ...ubroutines which minimize or maximize the Lagrangian individually in the primal or dual argument, cf. (1.2). For multistage, possibly stochastic, optimization problems expressed in the format of [1], =-=[2]-=-, and [6], such subroutines can easily be written in terms of the underlying data structure (without ever introducing R!). In obtaining our results about local rates of convergence, a mild condition o... |

14 |
An extension of Karmarkar's projective algorithm for convex quadratic programming
- Ye, Tse
- 1989
(Show Context)
Citation Context ...r complementarity problems [10], where U = lR n +, V = lR m + , and Q is not necessarily the zero matrix, are surveyed by Lin and Pang [11]. Other efforts in recent times have been made by Ye and Tse =-=[12]-=-, Monteiro and Adler [13], Goldfarb and Liu [14]. None of these approaches is consonant with the large-scale applications that attract our interest, because the structure in such applications is not w... |

11 |
Computational schemes for solving large-scale problems in extended linearquadratic programming
- Rockafellar
- 1990
(Show Context)
Citation Context ...gh dimensional. These models raise new computational challenges and possibilities. A foundation for numerical schemes in large-scale extended linear-quadratic programming has been laid in Rockafellar =-=[6]-=- and elaborated for problems in multistage format in Rockafellar [7]. The emphasis in [6] is on basic finite-envelope methods, which use duality in generating envelope approximations to the primal and... |

6 |
Methods for quadratic programming: a survey
- Pang
- 1983
(Show Context)
Citation Context ...ly if ū solves (P) and ¯v solves (Q), or equivalently, f(ū) = g(¯v). Current numerical methods in standard quadratic programming, and the somewhat more general area of linear complementarity problems =-=[10]-=-, where U = lR n +, V = lR m + , and Q is not necessarily the zero matrix, are surveyed by Lin and Pang [11]. Other efforts in recent times have been made by Ye and Tse [12], Monteiro and Adler [13], ... |

5 |
Modified proximal point algorithm for extended linear-quadratic programming
- Zhu
- 1992
(Show Context)
Citation Context ... character and relate to a host of primal-dual procedures in extended linearquadratic programming besides the ones developed here, cf. [1], [2], [6]. In particular, such questions are taken up in Zhu =-=[20]-=-. The supposition that line searches can be carried out exactly is an expedient to allow us to concentrate on more important matters for now. It is also in keeping with the exploration of finite termi... |

4 | Large-scale extended linear-quadratic programming and multistage optimization
- Rockafellar
- 1991
(Show Context)
Citation Context ...) instead of just projected gradients. These unprojected gradients are available through (1.11) and (1.12) (also (1.15) or (1.16)), and for multistage optimization problems in the pattern laid out in =-=[7]-=- they can still be calculated without having to invoke the gigantic R matrix. An earlier version of Algorithm 2 that we worked with did use the projected gradients exclusively, and it performed simila... |

3 |
An implementation of the Lagrangian finite generation method
- King
- 1988
(Show Context)
Citation Context ... finite sequence of separate minimizations or maximizations of the Lagrangian. These methods generalize the one originally proposed in [1] for two-stage stochastic programming and implemented by King =-=[8]-=- and Wagner [9]. They center *Received by the editors December ???, 1990; accepted for publication (in revised form) ??? ???, 1992. This work was supported in part by AFOSR grants 87-0281 and 89-0081 ... |

3 |
Stochastic Programming with Recourse Applied to Groundwater Quality Management, doctoral dissertation
- Wagner
- 1988
(Show Context)
Citation Context ...e of separate minimizations or maximizations of the Lagrangian. These methods generalize the one originally proposed in [1] for two-stage stochastic programming and implemented by King [8] and Wagner =-=[9]-=-. They center *Received by the editors December ???, 1990; accepted for publication (in revised form) ??? ???, 1992. This work was supported in part by AFOSR grants 87-0281 and 89-0081 and NSF grants ... |

1 |
A generalized approach to linear-quadratic programming
- Rockafellar
- 1986
(Show Context)
Citation Context ...ets in [1], [2], first used the concept in two-stage stochastic programming, where the primal dimension is low but the dual dimension is high. It was developed further in its own right in Rockafellar =-=[3]-=-, [4], and carried in the latter paper to the context of continuous-time optimal control. Discrete-time problems of optimal control, both deterministic and stochastic (i.e., multistage stochastic prog... |

1 |
with introduction by R.T. Rockafellar), DYNFGM: Dynamic Finite Generation
- Wright
- 1989
(Show Context)
Citation Context ...tructure enables us to use special routines in calculating f(u) and F (u), and on the other hand g(v) and G(v) [7]. For this purpose, and in implementing BFEM, we rely on code written by S. E. Wright =-=[25]-=- at the University of Washington. The problems have been obtained as discretized versions of certain continuoustime problems of extended linear-quadratic optimal control of the kind developed in Rocka... |