## On a modified subgradient algorithm for dual problems via sharp augmented Lagrangian (2006)

Venue: | Journal of Global Optimization |

Citations: | 3 - 1 self |

### BibTeX

@ARTICLE{Burachik06ona,

author = {Regina S. Burachik and Nergiz A. Ismayilova and Rafail N. Gasimov and C. Yalçın Kaya},

title = {On a modified subgradient algorithm for dual problems via sharp augmented Lagrangian},

journal = {Journal of Global Optimization},

year = {2006},

pages = {55--78}

}

### OpenURL

### Abstract

We study convergence properties of a modified subgradient algorithm, applied to the dual problem defined by the sharp augmented Lagrangian. The primal problem we consider is nonconvex and nondifferentiable, with equality constraints. We obtain primal and dual convergence results, as well as a condition for existence of a dual solution. Using a practical selection of the step-size parameters, we demonstrate the algorithm and its advantages on test problems, including an integer programming and an optimal control problem. Key words: Nonconvex programming; nonsmooth optimization; augmented Lagrangian; sharp Lagrangian; subgradient optimization.

### Citations

3267 | Wets, Variational Analysis - Rockafellar, J-B - 2009 |

716 |
Shetty: “Non-linear programming theory and algorithms
- Bazaraa, M
(Show Context)
Citation Context ...nd its extensions. Subgradient methods were introduced in the middle 60s, in the works of Demyanov [6], Poljak [16, 17, 18] and Shor [28, 29]. A detailed presentation of these methods can be found in =-=[3, 5, 13]-=- and the references therein. In this paper we study nonconvex optimization problems with equality constraints, where the cost and constraints are only required to be continuous; namely we do not pose ... |

273 |
Minimization Methods for Nondifferentiable Functions
- Shor
- 1985
(Show Context)
Citation Context .... A typical example of these techniques is the subgradient method and its extensions. Subgradient methods were introduced in the middle 60s, in the works of Demyanov [6], Poljak [16, 17, 18] and Shor =-=[28, 29]-=-. A detailed presentation of these methods can be found in [3, 5, 13] and the references therein. In this paper we study nonconvex optimization problems with equality constraints, where the cost and c... |

147 | Introduction to Optimization - Polyak - 1987 |

87 | Lagrange multipliers and optimality
- Rockafellar
- 1993
(Show Context)
Citation Context ...sharp Lagrangian duality scheme, when applied to problem (P ).Theorems1and2areproved(in a more general setting) in [20, Theorem 11.59], see also [2]. Note that Theorems 1, 2 and 3 have been proved in =-=[21]-=- for the classical quadratic augmented Lagrangian. The proof of Theorem 3 is a straightforward modification of the one given in [21, Theorem 6.4]. Theorem 1 Suppose that inf P is finite and consider t... |

38 |
The conjugate gradient method in extremal problems
- Polyak
- 1969
(Show Context)
Citation Context ...be used for solving it. A typical example of these techniques is the subgradient method and its extensions. Subgradient methods were introduced in the middle 60s, in the works of Demyanov [6], Poljak =-=[16, 17, 18]-=- and Shor [28, 29]. A detailed presentation of these methods can be found in [3, 5, 13] and the references therein. In this paper we study nonconvex optimization problems with equality constraints, wh... |

38 |
Minimization of Unsmooth Functionals
- Polyak
- 1969
(Show Context)
Citation Context ...be used for solving it. A typical example of these techniques is the subgradient method and its extensions. Subgradient methods were introduced in the middle 60s, in the works of Demyanov [6], Poljak =-=[16, 17, 18]-=- and Shor [28, 29]. A detailed presentation of these methods can be found in [3, 5, 13] and the references therein. In this paper we study nonconvex optimization problems with equality constraints, wh... |

32 |
Abstract convexity and global optimization
- Rubinov
- 2012
(Show Context)
Citation Context ...hese alternative ways was given in [1] by Azimov and Gasimov, who presented conditions which depend on the objective and constraint functions. Other conditions of this kind can be found in references =-=[9, 22, 24, 25, 26]-=-. We give below a new existence condition by using the dual sequence generated by the MSG algorithm using a general step-size sk. Inwhat follows, {zk} = {(uk,ck)} is the dual sequence generated by the... |

17 |
On the choice of step size in subgradient optimization
- Bazaraa, Sherali
- 1981
(Show Context)
Citation Context ...a value for the unknown H in the computation of sk is an important issue in subgradient methods. Note that any feasible solution of (P) provides an upper bound for H. According to Bazaraa and Sherali =-=[4]-=- H can be chosen as a convex combination of a fixed upper bound and the current best dual value. On the other hand, Sherali et al. [27]On a Modified Subgradient Algorithm for Dual Problems by Burachi... |

14 |
MINOS 5.4 User’s Guide. System Optimization
- Murtagh, Saunders
(Show Context)
Citation Context ... −1 denotes the initial guess used for the subproblem when k =0. Problem 1 The following nonconvex problem has become a common test problem (see [7]). It was originally studied by Murtagh and Saunders=-=[15]-=-. ⎧ ⎪⎨ P1 : ⎪⎩ min f0(x) =(x1 − 1) 2 +(x1 − x2) 2 +(x2 − x3) 3 +(x3 − x4) 4 +(x4 − x5) 4 subject to f1(x) =x1 + x 2 2 + x3 3 − 3 √ 2 − 2=0 f2(x) =x2 − x 2 3 + x4 − 2 √ 2+2=0 f3(x) =x1 x5 − 2=0 The rep... |

13 |
order sufficient conditions for time-optimal bang-bang control
- Maurer, Osmolovskii, et al.
(Show Context)
Citation Context ...ent Algorithm for Dual Problems by Burachik et al. 22 Problem 3 The following problem concerns finding a bang–bang constrained time-optimal control of the van der Pol system, which is also studied in =-=[10, 11, 14, 30]-=-. The dynamics of the van der Pol system are given by the ordinary differential equations ˙z1(t) = z2(t) , ˙z2(t) = −z1(t) − (z 2 1 (t) − 1) z2(t)+v(t) , (31) where ˙zi := dzi/dt, i =1, 2, the state v... |

11 |
and J.L.Noakes: Computational methods for time-optimal switching controls
- Kaya
(Show Context)
Citation Context ...ent Algorithm for Dual Problems by Burachik et al. 22 Problem 3 The following problem concerns finding a bang–bang constrained time-optimal control of the van der Pol system, which is also studied in =-=[10, 11, 14, 30]-=-. The dynamics of the van der Pol system are given by the ordinary differential equations ˙z1(t) = z2(t) , ˙z2(t) = −z1(t) − (z 2 1 (t) − 1) z2(t)+v(t) , (31) where ˙zi := dzi/dt, i =1, 2, the state v... |

10 | Decreasing functions with applications to penalization
- Rubinov, Glover, et al.
- 1999
(Show Context)
Citation Context ...ization problems. This justifies the quest for other kinds of augmented Lagrangians, which are able to provide solution algorithms for a broader family of constrained optimization problems. The works =-=[23, 24, 25, 26, 31]-=- study new kinds of Lagrangians, and their applications to different classes of constrained optimization problems. Specific applications of some of these new Lagrangians can be found in [8, 9]. The de... |

9 |
A nonlinear Lagrangian approach to constraint optimization problems
- Yang, Huang
(Show Context)
Citation Context ...ization problems. This justifies the quest for other kinds of augmented Lagrangians, which are able to provide solution algorithms for a broader family of constrained optimization problems. The works =-=[23, 24, 25, 26, 31]-=- study new kinds of Lagrangians, and their applications to different classes of constrained optimization problems. Specific applications of some of these new Lagrangians can be found in [8, 9]. The de... |

7 |
2002, Augmented Lagrangian duality and nondifferentiable optimization methods in nonconvex programming
- Gasimov
(Show Context)
Citation Context ...4, 25, 26, 31] study new kinds of Lagrangians, and their applications to different classes of constrained optimization problems. Specific applications of some of these new Lagrangians can be found in =-=[8, 9]-=-. The development of efficient methods for solving optimization problems depends on the duality relations provided by the augmented Lagrangian framework. The dual problem is typically a nonsmooth conv... |

7 |
Convergence of a Generalized Subgradient Method for Nondifferentiable Convex
- Kim, Ahn
- 1991
(Show Context)
Citation Context ...rgence of the dual values towards the optimal value H. The most commonly used step-sizes for the classical subgradient method are the infinite series rule and the known optimal value rule (see, e.g., =-=[12]-=-). As can be seen, the step-size we propose above for the MSG algorithm also requires the knowledge of the optimal value. Choosing a value for the unknown H in the computation of sk is an important is... |

6 | Computations and time-optimal controls
- Kaya, Noakes
- 1996
(Show Context)
Citation Context ...ent Algorithm for Dual Problems by Burachik et al. 22 Problem 3 The following problem concerns finding a bang–bang constrained time-optimal control of the van der Pol system, which is also studied in =-=[10, 11, 14, 30]-=-. The dynamics of the van der Pol system are given by the ordinary differential equations ˙z1(t) = z2(t) , ˙z2(t) = −z1(t) − (z 2 1 (t) − 1) z2(t)+v(t) , (31) where ˙zi := dzi/dt, i =1, 2, the state v... |

6 |
Dual estimate in multi-extremal problems
- Shor
- 1992
(Show Context)
Citation Context .... A typical example of these techniques is the subgradient method and its extensions. Subgradient methods were introduced in the middle 60s, in the works of Demyanov [6], Poljak [16, 17, 18] and Shor =-=[28, 29]-=-. A detailed presentation of these methods can be found in [3, 5, 13] and the references therein. In this paper we study nonconvex optimization problems with equality constraints, where the cost and c... |

3 |
Neuro-Dynamic programming
- Betsekas, Tsitsiklis
- 1995
(Show Context)
Citation Context ...nd its extensions. Subgradient methods were introduced in the middle 60s, in the works of Demyanov [6], Poljak [16, 17, 18] and Shor [28, 29]. A detailed presentation of these methods can be found in =-=[3, 5, 13]-=- and the references therein. In this paper we study nonconvex optimization problems with equality constraints, where the cost and constraints are only required to be continuous; namely we do not pose ... |

2 |
R.N.: On weak conjugacy, weak subdifferentials and duality with zero-gap in non-convex optimization
- Azimov, Gasimov
- 1999
(Show Context)
Citation Context ...thm solves the dual problem obtained with respect to the sharp Lagrangian. Note that sharp Lagrangians were introduced by Rockafellar and Wets in [20, Example 11.58]. This concept was also studied in =-=[1, 2]-=-, where it was used for establishing conditions for the existence of solutions and zero duality gap properties for some classes of nonconvex optimization problems with inequality constraints. One may ... |

2 |
Algorithms for Some Minimax Problems
- Demyanov
- 1968
(Show Context)
Citation Context ...hniques can be used for solving it. A typical example of these techniques is the subgradient method and its extensions. Subgradient methods were introduced in the middle 60s, in the works of Demyanov =-=[6]-=-, Poljak [16, 17, 18] and Shor [28, 29]. A detailed presentation of these methods can be found in [3, 5, 13] and the references therein. In this paper we study nonconvex optimization problems with equ... |

2 |
On convergence properties of a subgradient method
- Konnov
- 2003
(Show Context)
Citation Context ...nd its extensions. Subgradient methods were introduced in the middle 60s, in the works of Demyanov [6], Poljak [16, 17, 18] and Shor [28, 29]. A detailed presentation of these methods can be found in =-=[3, 5, 13]-=- and the references therein. In this paper we study nonconvex optimization problems with equality constraints, where the cost and constraints are only required to be continuous; namely we do not pose ... |

2 |
Iterative methods using Lagrange multipliers for solving extremal problems with constraints of the equation type. Z. Vychislitelnoy Matematiki i Matematicheskoy Fiziki
- Polyak
- 1969
(Show Context)
Citation Context ...be used for solving it. A typical example of these techniques is the subgradient method and its extensions. Subgradient methods were introduced in the middle 60s, in the works of Demyanov [6], Poljak =-=[16, 17, 18]-=- and Shor [28, 29]. A detailed presentation of these methods can be found in [3, 5, 13] and the references therein. In this paper we study nonconvex optimization problems with equality constraints, wh... |

2 |
2003, Lagrange-type functions in constrained optimization
- Rubinov, Yang, et al.
(Show Context)
Citation Context ...ization problems. This justifies the quest for other kinds of augmented Lagrangians, which are able to provide solution algorithms for a broader family of constrained optimization problems. The works =-=[23, 24, 25, 26, 31]-=- study new kinds of Lagrangians, and their applications to different classes of constrained optimization problems. Specific applications of some of these new Lagrangians can be found in [8, 9]. The de... |

2 |
2000, A variable target value method for nondifferentiable optimization
- Sherali, Choi, et al.
(Show Context)
Citation Context ...) provides an upper bound for H. According to Bazaraa and Sherali [4] H can be chosen as a convex combination of a fixed upper bound and the current best dual value. On the other hand, Sherali et al. =-=[27]-=-On a Modified Subgradient Algorithm for Dual Problems by Burachik et al. 15 recently proposed the so-called variable target value method which assumes no a priori knowledge regarding H. In the numeri... |

2 | 2002, Computations for time-optimal bang–bang control using a Lagrangian formulation
- Simakov, Kaya, et al.
(Show Context)
Citation Context |

1 |
2002, Stability and Duality of Nonconvex Problems via Augmented Lagrangian
- Azimov, Gasimov
(Show Context)
Citation Context ...thm solves the dual problem obtained with respect to the sharp Lagrangian. Note that sharp Lagrangians were introduced by Rockafellar and Wets in [20, Example 11.58]. This concept was also studied in =-=[1, 2]-=-, where it was used for establishing conditions for the existence of solutions and zero duality gap properties for some classes of nonconvex optimization problems with inequality constraints. One may ... |

1 |
2004, On augmented Lagrangians for optimization problems with a single constraint
- Gasimov, Rubinov
(Show Context)
Citation Context ...hese alternative ways was given in [1] by Azimov and Gasimov, who presented conditions which depend on the objective and constraint functions. Other conditions of this kind can be found in references =-=[9, 22, 24, 25, 26]-=-. We give below a new existence condition by using the dual sequence generated by the MSG algorithm using a general step-size sk. Inwhat follows, {zk} = {(uk,ck)} is the dual sequence generated by the... |

1 |
The nonlinear and augmented Lagrangians for nonconvex optimization problems with a single constraint
- Rubinov, Gasimov
(Show Context)
Citation Context |

1 | 2003, Strictly increasing positively homogeneous functions with applications to exact penalization
- Rubinov, Gasimov
(Show Context)
Citation Context |