## A class of inexact variable metric proximal point algorithms (2008)

Venue: | SIAM Journal on Optimization |

Citations: | 10 - 5 self |

### BibTeX

@ARTICLE{Parente08aclass,

author = {L. A. Parente and P. A. Lotito and M. V. Solodov},

title = {A class of inexact variable metric proximal point algorithms},

journal = {SIAM Journal on Optimization},

year = {2008},

pages = {240--260}

}

### OpenURL

### Abstract

Abstract. For the problem of solving maximal monotone inclusions, we present a rather general class of algorithms, which contains hybrid inexact proximal point methods as a special case and allows for the use of a variable metric in subproblems. The global convergence and local linear rate of convergence are established under standard assumptions. We demonstrate the advantage of variable metric implementation in the case of solving systems of smooth monotone equations by the proximal Newton method.

### Citations

3760 | Convex Analysis
- Rockafellar
- 1970
(Show Context)
Citation Context ...). As is well known, many important problems can be cast in this framework. Some examples are convex optimization, min-max problems, and monotone variational inequalities over convex sets; see, e.g., =-=[23]-=-. Given some z k ∈ R n , the current approximation to a solution of (1.1), the proximal point iteration [19, 22] generates z k+1 as the solution of the regularized subproblem (1.2) 0 ∈ ckT (z)+z − z k... |

1045 | Numerical methods for unconstrained optimization and nonlinear equations - Dennis, Schnabel - 1996 |

470 |
Convex Analysis and Minimization Algorithms
- Hiriart-Urruty, Lemaréchal
- 1993
(Show Context)
Citation Context ...techniques, such as splitting methods for variational problems (e.g., [18, 31, 13, 33, 34, 24]), the methods of multipliers (e.g., [21, 15]), and bundle methods for nonsmooth optimization (see, e.g., =-=[16, 3]-=-), to name a few. In computational context, it is important to handle approximate solutions of subproblems; this will be discussed a little further. Also, it is attractive to allow for the use of a va... |

306 |
Monotone operators and the proximal point algorithm
- Rockafellar
- 1976
(Show Context)
Citation Context ...ation, min-max problems, and monotone variational inequalities over convex sets; see, e.g., [23]. Given some z k ∈ R n , the current approximation to a solution of (1.1), the proximal point iteration =-=[19, 22]-=- generates z k+1 as the solution of the regularized subproblem (1.2) 0 ∈ ckT (z)+z − z k , where ck > 0 is the regularization parameter. As is well known, the proximal point method serves as a basis f... |

198 | On the Douglas–Rachford splitting method and the proximal point algorithm for maximal monotone operators
- Eckstein, Bertsekas
- 1992
(Show Context)
Citation Context ...arameter. As is well known, the proximal point method serves as a basis for developing and analyzing various useful computational techniques, such as splitting methods for variational problems (e.g., =-=[18, 31, 13, 33, 34, 24]-=-), the methods of multipliers (e.g., [21, 15]), and bundle methods for nonsmooth optimization (see, e.g., [16, 3]), to name a few. In computational context, it is important to handle approximate solut... |

143 |
Augmented Lagrangians and applications of the proximal point algorithm in convex programming
- Rockafellar
- 1976
(Show Context)
Citation Context ...as a basis for developing and analyzing various useful computational techniques, such as splitting methods for variational problems (e.g., [18, 31, 13, 33, 34, 24]), the methods of multipliers (e.g., =-=[21, 15]-=-), and bundle methods for nonsmooth optimization (see, e.g., [16, 3]), to name a few. In computational context, it is important to handle approximate solutions of subproblems; this will be discussed a... |

124 |
Splitting algorithms for the sum of two nonlinear operators
- Lions, Mercier
- 1979
(Show Context)
Citation Context ...arameter. As is well known, the proximal point method serves as a basis for developing and analyzing various useful computational techniques, such as splitting methods for variational problems (e.g., =-=[18, 31, 13, 33, 34, 24]-=-), the methods of multipliers (e.g., [21, 15]), and bundle methods for nonsmooth optimization (see, e.g., [16, 3]), to name a few. In computational context, it is important to handle approximate solut... |

91 |
Numerical Optimization: Theoretical and Practical Aspects
- Bonnans, Gilbert, et al.
- 2006
(Show Context)
Citation Context ...techniques, such as splitting methods for variational problems (e.g., [18, 31, 13, 33, 34, 24]), the methods of multipliers (e.g., [21, 15]), and bundle methods for nonsmooth optimization (see, e.g., =-=[16, 3]-=-), to name a few. In computational context, it is important to handle approximate solutions of subproblems; this will be discussed a little further. Also, it is attractive to allow for the use of a va... |

72 |
Applications of a splitting algorithm to decomposition in convex programming and variational inequalities
- Tseng
- 1991
(Show Context)
Citation Context ...arameter. As is well known, the proximal point method serves as a basis for developing and analyzing various useful computational techniques, such as splitting methods for variational problems (e.g., =-=[18, 31, 13, 33, 34, 24]-=-), the methods of multipliers (e.g., [21, 15]), and bundle methods for nonsmooth optimization (see, e.g., [16, 3]), to name a few. In computational context, it is important to handle approximate solut... |

68 |
Applications of the method of multipliers to variational inequalities
- Gabay
- 1983
(Show Context)
Citation Context ...as a basis for developing and analyzing various useful computational techniques, such as splitting methods for variational problems (e.g., [18, 31, 13, 33, 34, 24]), the methods of multipliers (e.g., =-=[21, 15]-=-), and bundle methods for nonsmooth optimization (see, e.g., [16, 3]), to name a few. In computational context, it is important to handle approximate solutions of subproblems; this will be discussed a... |

64 |
Régularization d’inéquations variationelles par approximations successive, Revue Française de Informatique at Recherche Opérationelle 2
- Martinet
- 1970
(Show Context)
Citation Context ...ation, min-max problems, and monotone variational inequalities over convex sets; see, e.g., [23]. Given some z k ∈ R n , the current approximation to a solution of (1.1), the proximal point iteration =-=[19, 22]-=- generates z k+1 as the solution of the regularized subproblem (1.2) 0 ∈ ckT (z)+z − z k , where ck > 0 is the regularization parameter. As is well known, the proximal point method serves as a basis f... |

51 | A modified forward-backward splitting method for maximal monotone mappings
- Tseng
(Show Context)
Citation Context |

42 | Variable metric bundle methods: from conceptual to implementable forms
- Lemaréchal, Sagastizábal
- 1997
(Show Context)
Citation Context ...-related schemes in the literature that use variable metrics typically deal only with the special case of optimization, i.e., the case where the operator T is the subdifferential of a convex function =-=[2, 20, 17, 10]-=-. To our knowledge, the exception is [7] and some of the subsequent results [8, 9]. We note that our use of a variable metric is different from [7], where (exact) iteration is of the form z k+1 = z k ... |

39 | A hybrid projection-proximal point algorithm
- Solodov, Svaiter
- 1999
(Show Context)
Citation Context ...in our application to solving systems of monotone equations, discussed in section 5. To handle approximate solutions, we shall use an extension to the variable metric setting of the rules proposed in =-=[27, 26]-=- and unified in [30]. In those algorithms, the relative error in the approximation needs only to be bounded (above, by one), which is a numerically sound requirement, and inexact values of the operato... |

33 | Enlargement of monotone operators with applications to variational inequalities
- Burachik, Iusem, et al.
- 1997
(Show Context)
Citation Context ...tion) parameter, by ‖·‖M we denote the norm induced by a symmetric positive definite matrix M ∈M n ++, i.e., ‖z‖M = √ 〈z,Mz〉, and T ε : R n ⇒ R n is the ε-enlargement of a maximal monotone operator T =-=[5, 6]-=-, defined as T ε (z) :={v ∈ R n |〈w − v, y − z〉 ≥−ε ∀y ∈ R n , ∀w ∈ T (y)}, ε≥ 0. We note that, to check the above criterion, one does not need to invert the matrix Mk, as will be explained in what fo... |

28 | A hybrid approximate extragradient-proximal point algorithm using the enlargement of a maximal monotone operator
- Solodov, Svaiter
- 1999
(Show Context)
Citation Context ...in our application to solving systems of monotone equations, discussed in section 5. To handle approximate solutions, we shall use an extension to the variable metric setting of the rules proposed in =-=[27, 26]-=- and unified in [30]. In those algorithms, the relative error in the approximation needs only to be bounded (above, by one), which is a numerically sound requirement, and inexact values of the operato... |

27 | A unified framework for some inexact proximal point algorithms
- Solodov, Svaiter
- 2001
(Show Context)
Citation Context ...lving systems of monotone equations, discussed in section 5. To handle approximate solutions, we shall use an extension to the variable metric setting of the rules proposed in [27, 26] and unified in =-=[30]-=-. In those algorithms, the relative error in the approximation needs only to be bounded (above, by one), which is a numerically sound requirement, and inexact values of the operator T are allowed, whi... |

26 | A variable metric proximal point algorithm for monotone operators
- Burke, Qian
- 1998
(Show Context)
Citation Context ... matrix Mk, as will be explained in what follows. The presented approximation rule is constructive and has advantages in some situations, when compared to the original [22] (and its variations, e.g., =-=[32, 11, 7]-=-), where essentially one has εk = 0 and ∑∞ k=0 ‖δk‖ < ∞ (in the setting of Mk = I). We refer the reader to [26, 29, 28, 24] for some applications where the relative-error criterion appears useful. It ... |

22 | A proximal point algorithm for the variational inequality problem in Banach spaces - Burachik, Scheimberg |

21 | A globally convergence inexact Newton method for systems of monotone equations
- Solodov, Svaiter
- 1998
(Show Context)
Citation Context ...entarity problems [14]. We start with describing the method and giving its theoretical justification and then report on our numerical experiments. 5.1. Description and justification of the method. In =-=[25, 29]-=-, it has been shown that hybrid inexact proximal point schemes (with a fixed metric) can be used to construct Newton methods for monotone problems with a very attractive combination of global and loca... |

21 |
Applications of the method of partial inverses to convex programming: decomposition
- Spingarn
- 1985
(Show Context)
Citation Context |

20 | A preconditioning proximal Newton method for nondifferentiable convex optimization
- Qi, Chen
- 1997
(Show Context)
Citation Context ...-related schemes in the literature that use variable metrics typically deal only with the special case of optimization, i.e., the case where the operator T is the subdifferential of a convex function =-=[2, 20, 17, 10]-=-. To our knowledge, the exception is [7] and some of the subsequent results [8, 9]. We note that our use of a variable metric is different from [7], where (exact) iteration is of the form z k+1 = z k ... |

19 | A truly globally convergent Newton-type method for the monotone nonlinear complementarity problem
- Solodov, Svaiter
(Show Context)
Citation Context ...rror in the approximation needs only to be bounded (above, by one), which is a numerically sound requirement, and inexact values of the operator T are allowed, which is useful in various applications =-=[29, 28, 24]-=-. Specifically, to solve (1.3) approximately, the task would be to compute a triplet (ˆz k , ˆv k ,εk) ∈ Rn × Rn × R+ such that { k εk k ˆv ∈ T (ˆz ), ckMkˆv k +ˆz k − zk = δk , ‖δ k ‖ 2 M −1 +2ckεk ≤... |

17 |
Projection algorithms: results and open problems
- Bauschke
- 2001
(Show Context)
Citation Context ...ect to the set T −1 (0) (for the given norm). For Fejér-monotone sequences, linear convergence of {dist(z k ,T −1 (0))} is equivalent to the linear convergence rate of {z k } to its limit (see, e.g., =-=[1]-=-). By the same argument as above, if ck →∞and σk → 0, then μk → 0, and (4.21) shows a superlinear convergence rate. Assume now that the condition (4.9) holds. Then (4.22) Define 1 (1 + ηk) dist(z,T −1... |

17 | Proximal quasi-Newton methods for nondifferentiable convex optimization
- Chen, Fukushima
- 1999
(Show Context)
Citation Context ...-related schemes in the literature that use variable metrics typically deal only with the special case of optimization, i.e., the case where the operator T is the subdifferential of a convex function =-=[2, 20, 17, 10]-=-. To our knowledge, the exception is [7] and some of the subsequent results [8, 9]. We note that our use of a variable metric is different from [7], where (exact) iteration is of the form z k+1 = z k ... |

15 | Error bounds for proximal point subproblems and associated inexact proximal point algorithms
- Solodov, Svaiter
- 2000
(Show Context)
Citation Context ...rror in the approximation needs only to be bounded (above, by one), which is a numerically sound requirement, and inexact values of the operator T are allowed, which is useful in various applications =-=[29, 28, 24]-=-. Specifically, to solve (1.3) approximately, the task would be to compute a triplet (ˆz k , ˆv k ,εk) ∈ Rn × Rn × R+ such that { k εk k ˆv ∈ T (ˆz ), ckMkˆv k +ˆz k − zk = δk , ‖δ k ‖ 2 M −1 +2ckεk ≤... |

13 |
A family of variable metric proximal point methods
- Bonnans, Gilbert, et al.
- 1995
(Show Context)
Citation Context |

12 | M.C.: Smooth methods of multipliers for complementarity problems
- Eckstein, Ferris
- 1999
(Show Context)
Citation Context ...solving a system of monotone differentiable equations (5.1) F (x) =0, where F : Rn → Rn . Problems of this type appear, for example, in smooth multiplier methods for monotone complementarity problems =-=[14]-=-. We start with describing the method and giving its theoretical justification and then report on our numerical experiments. 5.1. Description and justification of the method. In [25, 29], it has been ... |

11 |
Coupling the proximal point algorithm with approximation methods
- Cominetti
- 1997
(Show Context)
Citation Context ... matrix Mk, as will be explained in what follows. The presented approximation rule is constructive and has advantages in some situations, when compared to the original [22] (and its variations, e.g., =-=[32, 11, 7]-=-), where essentially one has εk = 0 and ∑∞ k=0 ‖δk‖ < ∞ (in the setting of Mk = I). We refer the reader to [26, 29, 28, 24] for some applications where the relative-error criterion appears useful. It ... |

10 | On the superlinear convergence of the variable metric proximal point algorithm using Broyden and BFGS matrix secant updating
- Burke, Qian
(Show Context)
Citation Context ...pecial case of optimization, i.e., the case where the operator T is the subdifferential of a convex function [2, 20, 17, 10]. To our knowledge, the exception is [7] and some of the subsequent results =-=[8, 9]-=-. We note that our use of a variable metric is different from [7], where (exact) iteration is of the form z k+1 = z k + Mk((I + ckT ) −1 − I)z k . The exact iteration of solving (1.3) can be written a... |

10 |
The perturbed Proximal Point Algorithm and some of its Applications
- Tossings
- 1994
(Show Context)
Citation Context ... matrix Mk, as will be explained in what follows. The presented approximation rule is constructive and has advantages in some situations, when compared to the original [22] (and its variations, e.g., =-=[32, 11, 7]-=-), where essentially one has εk = 0 and ∑∞ k=0 ‖δk‖ < ∞ (in the setting of Mk = I). We refer the reader to [26, 29, 28, 24] for some applications where the relative-error criterion appears useful. It ... |

9 |
A class of decomposition methods for convex optimization and monotone variational inclusions via the hybrid inexact proximal point framework
- Solodov
- 2004
(Show Context)
Citation Context |

6 |
ε-Enlargements of maximal monotone operators: Theory and Application
- Burachik, Sagastizábal, et al.
- 1998
(Show Context)
Citation Context ...tion) parameter, by ‖·‖M we denote the norm induced by a symmetric positive definite matrix M ∈M n ++, i.e., ‖z‖M = √ 〈z,Mz〉, and T ε : R n ⇒ R n is the ε-enlargement of a maximal monotone operator T =-=[5, 6]-=-, defined as T ε (z) :={v ∈ R n |〈w − v, y − z〉 ≥−ε ∀y ∈ R n , ∀w ∈ T (y)}, ε≥ 0. We note that, to check the above criterion, one does not need to invert the matrix Mk, as will be explained in what fo... |

2 | On the local super-linear convergence of a matrix secant implementation of the variable metric proximal point algorithm for monotone operators
- Burke, Qian
- 1999
(Show Context)
Citation Context ...pecial case of optimization, i.e., the case where the operator T is the subdifferential of a convex function [2, 20, 17, 10]. To our knowledge, the exception is [7] and some of the subsequent results =-=[8, 9]-=-. We note that our use of a variable metric is different from [7], where (exact) iteration is of the form z k+1 = z k + Mk((I + ckT ) −1 − I)z k . The exact iteration of solving (1.3) can be written a... |