## An iterative regularization method for total variation-based image restoration (2005)

### Cached

### Download Links

Venue: | Simul |

Citations: | 101 - 20 self |

### BibTeX

@ARTICLE{Osher05aniterative,

author = {Stanley Osher and Martin Burger and Donald Goldfarb and Jinjun Xu and Wotao Yin},

title = {An iterative regularization method for total variation-based image restoration},

journal = {Simul},

year = {2005},

volume = {4},

pages = {460--489}

}

### Years of Citing Articles

### OpenURL

### Abstract

Abstract. We introduce a new iterative regularization procedure for inverse problems based on the use of Bregman distances, with particular focus on problems arising in image processing. We are motivated by the problem of restoring noisy and blurry images via variational methods by using total variation regularization. We obtain rigorous convergence results and effective stopping criteria for the general procedure. The numerical results for denoising appear to give significant improvement over standard models, and preliminary results for deblurring/denoising are very encouraging.

### Citations

1500 |
Non linear total variation based noise removal algorithms
- Rudin, Fatemi
- 1992
(Show Context)
Citation Context ...nd v is the noise. This problem has a very long history (cf. [30]). One of the most successful and popular techniques for approximating the solution of this problem is due to Rudin, Osher, and Fatemi =-=[38]-=- and is defined as follows: � u = arg min |u|BV + λ�f − u� u∈BV (Ω) 2 L2 � (1.1) for some scale parameter λ>0, where BV (Ω) denotes the space of functions with bounded variation on Ω and |·|denotes th... |

951 | Sethian Fronts propagating with curvature-dependent speed: Algorithms base on hamilton-Jacabi formulations
- Osher, A
- 1988
(Show Context)
Citation Context ...ng the BV seminorm. 2. Using geometry and iterative regularization. Our present work has several immediate antecedents. In [44], Tasdizen et al. processed deformable surfaces via the level set method =-=[34]-=-. The idea was used to (a) first process the unit normals to a given initial surface and (b) deform the surface so as to simultaneously process it and fit it to the previously computed surface. The re... |

481 |
Regularization of Inverse Problems
- Engl, Hanke, et al.
- 1996
(Show Context)
Citation Context ...roach the true solution until the residual in the iteration drops below the noise level. The result of Theorem 3.5 yields a natural stopping rule, the so-called generalized discrepancy principle (cf. =-=[21]-=-), which consists in stopping the iteration at the index k∗ = k∗(δ, f) given by (3.15) k∗ = max{k ∈ N | H(uk,f) ≥ τδ 2 }, where τ>1. Note that due to the monotone decrease of H(uk,f), which is guarant... |

467 |
Convex analysis and variational problems
- Ekeland, Temam
- 1976
(Show Context)
Citation Context ...ularization functional (for total variation regularization we have J(u)= � |∇u|) and the fitting functional H is convex nonnegative with respect to u for fixed f. As usual for convex functionals (cf. =-=[20]-=-) we shall denote the subdifferential of J at a point u by ∂J(u):={p ∈ BV (Ω) ∗ | J(v) ≥ J(u)+〈p, v − u〉 ∀v ∈ BV (Ω)}. After initializing u0 = 0 and p0 =0∈ ∂J(u0), the iterative procedure is given by ... |

398 | Image inpainting
- Bertalmio, Sapiro, et al.
- 2000
(Show Context)
Citation Context ...e representations and total variation regularization. One of the many reasons to separate cartoon from texture is to improve image inpainting algorithms. See [5] for a successful approach to this and =-=[4]-=- for a pioneering paper on this subject. We also mention here that using duality, Chambolle [11] constructed an algorithm solving for v directly in a way that simplifies the calculations needed to sol... |

285 |
The relaxation method of finding the common points of convex sets and its application to the solution of problems in convex programming
- Bregman
- 1967
(Show Context)
Citation Context ...restoration and its generalizations. Instead of stopping after recovering the minimizer u in (1.1), we call this solution u1 and use it to compute u2, u3, etc. This is done using the Bregman distance =-=[6]-=-, which we shall define in our context in section 3.1. If we call D(u, v) the Bregman distance between u and v associated with the functional J, our algorithm designed to improve (1.1) is (1.2) uk = a... |

266 |
Image Recovery via Total Variation Minimization and Related
- Chambolle, Lions
- 1997
(Show Context)
Citation Context ... also referred to as the total variation of u. We call this variational problem the Rudin–Osher–Fatemi (ROF) model. It has been used and analyzed by several authors in several different contexts (cf. =-=[1, 8, 12, 17, 31, 36, 47]-=-). Also, in [12] and subsequently in [31, 32, 33, 36] the “staircasing” effect of this model was analyzed. No completely satisfying remedy has yet been found; e.g., see our results in Figures 1(c)–1(f... |

226 |
Computational methods for inverse problems
- Vogel
(Show Context)
Citation Context ... also referred to as the total variation of u. We call this variational problem the Rudin–Osher–Fatemi (ROF) model. It has been used and analyzed by several authors in several different contexts (cf. =-=[1, 8, 12, 17, 31, 36, 47]-=-). Also, in [12] and subsequently in [31, 32, 33, 36] the “staircasing” effect of this model was analyzed. No completely satisfying remedy has yet been found; e.g., see our results in Figures 1(c)–1(f... |

213 |
Oscillating patterns in image processing and nonlinear evolution equations, The fifteenth Dean Jacqueline B. Lewis memorial lectures
- Meyer
- 2001
(Show Context)
Citation Context ... a noisy image f :Ω→ R, where Ω is a bounded open subset of R 2 ,we want to obtain a decomposition, f = u + v, where u is the true signal and v is the noise. This problem has a very long history (cf. =-=[30]-=-). One of the most successful and popular techniques for approximating the solution of this problem is due to Rudin, Osher, and Fatemi [38] and is defined as follows: � u = arg min |u|BV + λ�f − u� u∈... |

172 | A nonlinear primal-dual method for total variation-based image restoration
- Chan, Golub, et al.
- 1996
(Show Context)
Citation Context ...on here that using duality, Chambolle [11] constructed an algorithm solving for v directly in a way that simplifies the calculations needed to solve (1.1), (1.8), and (1.12). Duality was also used in =-=[14]-=- to solve (1.1). We note that for each choice of λ there is a δ such that problem (1.1) is equivalent to the constrained minimization problem � (1.15) u = arg min u∈BV (Ω) |u|BV subject to �f − u� 2 L... |

157 | Modeling textures with total variation minimization and oscillating patterns in image processing
- Vese, Osher
- 2003
(Show Context)
Citation Context ... in the functional. However, it can be solved effectively as the minimization of a smooth function subject to constraints and, in particular, as a second-order cone program [23]. (1.8) Vese and Osher =-=[45]-=- approximated Meyer’s model by �� (u, g) = arg min (u,g) � |∇u| + λ |f − u −∇·g| 2 �� + μ |g| p � 1 � p with p ≥ 1 and λ, μ > 0. As λ, p → ∞, (1.8) approaches Meyer’s model. The results displayed in [... |

141 |
Total variation based image restoration with free local constraints
- Rudin, Osher
- 1994
(Show Context)
Citation Context ...the edges are sharper with our new model. We will also show how our new procedure can be used for other image restoration tasks, e.g., restoring blurry and noisy images, thus improving the results of =-=[39]-=-. The decomposition in this case becomes f = Au + v, where A is a given compact operator, often a convolution using, e.g., a Gaussian kernel. If A is not known, this becomes a blind deconvolution prob... |

141 | Image decomposition via the combination of sparse representations and a variational approach
- Starck, Elad, et al.
- 2005
(Show Context)
Citation Context ...parating cartoon from texture as the Vese–Osher model [45]. See also [3] for an explanation of this phenomenon. Additional work on a cartoon/texture decomposition was done in [2] using duality and in =-=[41]-=- using a combination of sparse representations and total variation regularization. One of the many reasons to separate cartoon from texture is to improve image inpainting algorithms. See [5] for a suc... |

127 | Total variation blind deconvolution
- Chan, Wong
- 1998
(Show Context)
Citation Context ...omposition in this case becomes f = Au + v, where A is a given compact operator, often a convolution using, e.g., a Gaussian kernel. If A is not known, this becomes a blind deconvolution problem. See =-=[16]-=- for an interesting approach to blind deconvolution, also minimizing functionals involving the BV seminorm. 2. Using geometry and iterative regularization. Our present work has several immediate antec... |

121 | Image decomposition and restoration using total variation minimization and the H−1 norm. Multiscale Model
- Osher, Solé, et al.
(Show Context)
Citation Context ...) If g = 0, then �∇(u − f)�q ≤ μ p , where q = 2λ p − 1 ; (1.10) Both u =0,g =0⇐⇒ �f�∗ ≤ 1 2λ , �∇f�q ≤ μ 2λ . (1.11) Yet another approximation to (1.8) was later constructed by Osher, Solé, and Vese =-=[35]-=-: (1.12) � u = arg min u∈BV (Ω) |u|BV + λ�∇Δ −1 (f − u)� 2 � ; see [35] for details. The (L 2 ) 2 -fitting term used in the ROF model is replaced by an (H −1 ) 2 -fitting term. This is also an f = u +... |

103 | Aspects of total variation regularized L 1 function approximation - Chan |

73 |
Dual norms and image decomposition models
- Aujol, Chambolle
- 2005
(Show Context)
Citation Context ...job at denoising images than the ROF model (although there is more computational effort involved), but it does not do as well in separating cartoon from texture as the Vese–Osher model [45]. See also =-=[3]-=- for an explanation of this phenomenon. Additional work on a cartoon/texture decomposition was done in [2] using duality and in [41] using a combination of sparse representations and total variation r... |

72 | Edge-preserving and scale-dependent properties of total variation Regularization
- Strong, Chan
(Show Context)
Citation Context ...so apply to the radially symmetric piecewise constant image f (2.4) if radially symmetric noise that is not too large is added to it. This follows from an analysis of the ROF model by Strong and Chan =-=[42]-=-. Strong and Chan present numerical results that show that their analytical results predict quite well the actual performance of ROF, even on digital images with no radial symmetry. Chambolle [11] has... |

58 | Local strong homogeneity of a regularized estimator - Nikolova |

53 | A multiscale image representation using hierarchical (BV,L2) decompositions. Multiscale Model
- Tadmor, Nezzar, et al.
(Show Context)
Citation Context ...ation with a Bregman distance (in the generalized sense) corresponding to a nondifferentiable functional, the total variation.s462 OSHER, BURGER, GOLDFARB, XU, AND YIN We also note that previously in =-=[43]-=- the authors constructed a sequence of approximations {uk} using ROF with a quite different approach, used more to decompose images than to restore them. We comment on this in section 3.5. The ideal r... |

52 |
Analysis of total variation penalty methods: Inverse Problems
- Acar, Vogel
- 1994
(Show Context)
Citation Context ... also referred to as the total variation of u. We call this variational problem the Rudin–Osher–Fatemi (ROF) model. It has been used and analyzed by several authors in several different contexts (cf. =-=[1, 8, 12, 17, 31, 36, 47]-=-). Also, in [12] and subsequently in [31, 32, 33, 36] the “staircasing” effect of this model was analyzed. No completely satisfying remedy has yet been found; e.g., see our results in Figures 1(c)–1(f... |

49 | Second-order cone programming methods for total variation-based image restoration
- Goldfarb, Yin
(Show Context)
Citation Context ...ness of both terms involved in the functional. However, it can be solved effectively as the minimization of a smooth function subject to constraints and, in particular, as a second-order cone program =-=[23]-=-. (1.8) Vese and Osher [45] approximated Meyer’s model by �� (u, g) = arg min (u,g) � |∇u| + λ |f − u −∇·g| 2 �� + μ |g| p � 1 � p with p ≥ 1 and λ, μ > 0. As λ, p → ∞, (1.8) approaches Meyer’s model.... |

43 | Proximal Minimization Methods with Generalized Bregman Functions
- Kiwiel
- 1997
(Show Context)
Citation Context ...ine the (nonnegative) quantity D p (u, v) ≡ D p J (u, v) ≡ J(u) − J(v) −〈p, u − v〉,s470 OSHER, BURGER, GOLDFARB, XU, AND YIN which is known as a generalized Bregman distance associated with J(·) (cf. =-=[6, 18, 28]-=- for an extension to nonsmooth functions). For simplicity, we will drop the dependence on J(·) from the notation D p (u, v) in the following. J For a continuously differentiable functional, there is a... |

42 |
Convergence analysis of a proximal-like minimization algorithm using Bregman functions
- Chen, Teboulle
- 1993
(Show Context)
Citation Context ...the first step can be put into this iterative framework by choosing initial values u0 = 0 and v0 = 0. We shall give precise reasons why this is a good procedure, using the concept of Bregman distance =-=[6, 18]-=- from convex programming, in the next section. Specifically, we are proposing the following iterative regularization procedure: • Initialize: u0 = 0 and v0 =0. • For k =0,1,2,...: compute uk+1 as a mi... |

39 | Convergence rates of convex variational regularization
- Burger, Osher
(Show Context)
Citation Context ...ciated with the total variation is difficult to interpret. However, at least for some cases the convergence of Bregman distances can be used to interpret the convergence speed of discontinuities (cf. =-=[7]-=-). 3.2. Well-definedness of the iterates. In the following we show that the iterative procedure in Algorithm 1 is well defined, i.e., that Qk has a minimizer uk and that we may find a suitable subgrad... |

38 |
High-order total variation-based image restoration
- Chan, Marquina, et al.
(Show Context)
Citation Context ...esting, in particular, for deblurring and for more general fitting functionals as outlined below. • Derivatives of bounded variation: Another obvious generalization considered by several authors (cf. =-=[15, 27]-=-) is to use the bounded variation of ∇u, i.e., � J(u)= |D 2 u|, Ω where D2u denotes the Hessian of u, or even more general functionals of the form � J(u)= ϕ(u, ∇u, D 2 u), Ω with convex ϕ : R × R 2 × ... |

35 | Aspects of total variation regularized L1 function approximation. CAM-Report 04-07 - Chan, Esedoglu - 2004 |

33 | Numerical methods for pharmonic flows and applications to image processing
- Vese, Osher
(Show Context)
Citation Context ... u2 = arg min u∈BV (Ω) (f − u) 2 � . This minimization procedure attempts to match normals as well as grey-level values. In [29] the denoised normal �n1 was computed by using a one-harmonic map as in =-=[46]-=-: �� �n1 = arg min |�n|=1 � � |∇�n| + λ �n − ∇f � � 2 . |∇f| Unlike all the other methods discussed in this paper, this is not a convex minimization problem, and it does not produce an image u1 satisf... |

32 | A regularizing Levenberg-Marquardt scheme with applications to inverse groundwater problems, Inverse Problems 13
- Hanke
- 1997
(Show Context)
Citation Context ...−1 and transformed output ˜ f = Af. 3.3. Convergence analysis. We shall now study some convergence properties of the iterative regularization process. Our analysis below is motivated by that of Hanke =-=[25]-=-, who analyzed Levenberg–Marquardt methods for ill-posed problems (also related to nonstationary iterative Tikhonov regularization (cf. [26, 24]) and inverse scale space methods (cf. [40])), which tur... |

32 | Noise removal using smoothed normals and surface fitting
- Lysaker, Osher, et al.
(Show Context)
Citation Context ...m the surface so as to simultaneously process it and fit it to the previously computed surface. The results were visually very pleasing, but no detailed theoretical analysis has yet been obtained. In =-=[29]-=-, Lysaker, Osher, and Tai borrowed the basic idea discussed above and applied it to images as follows. (This is purely formal analysis; see [29] for implementation details.) • Step 1: Given f, compute... |

30 |
Structural properties of solutions to total variation regularization problems
- Ring
(Show Context)
Citation Context |

27 | Decomposition of images by the anisotropic Rudin-OsherFatemi model
- Esedoḡlu, Osher
(Show Context)
Citation Context ... such as indicator functions of squares as minimizers, one can use anisotropic regularization functionals of the form � J(u)= G(∇u), with G : R2 → R + being a continuous one-homogeneous function (cf. =-=[22]-=-). An example of particular interest is G(∇u)=|ux| + |uy|. Of course, we can also use functions, which are not one-homogeneous, such as G(∇u)=�∇u�2 , thus including standard Tikhonov-type regularizati... |

27 |
Weakly constrained minimization. application to the estimation of images and signals involving constant regions
- Nikolova
(Show Context)
Citation Context ...iational problem the Rudin–Osher–Fatemi (ROF) model. It has been used and analyzed by several authors in several different contexts (cf. [1, 8, 12, 17, 31, 36, 47]). Also, in [12] and subsequently in =-=[31, 32, 33, 36]-=- the “staircasing” effect of this model was analyzed. No completely satisfying remedy has yet been found; e.g., see our results in Figures 1(c)–1(f) in this work for 0 ≤ x ≤ 120. In spite of this phen... |

20 |
Regularization by Functions of Bounded Variation and Applications to Image Enhancement
- Casas, Kunisch, et al.
(Show Context)
Citation Context |

16 | Regularization of linear least squares problems by total bounded variation
- Chavent, Kunisch
- 1997
(Show Context)
Citation Context |

15 |
Inverse scale space theory for inverse problems
- Scherzer, Groetsch
- 2001
(Show Context)
Citation Context ...hat of Hanke [25], who analyzed Levenberg–Marquardt methods for ill-posed problems (also related to nonstationary iterative Tikhonov regularization (cf. [26, 24]) and inverse scale space methods (cf. =-=[40]-=-)), which turns out to be a special case of our iterative regularization strategy when using a quadratic regularization functional J(u)=�u�2 for some Hilbert space norm. First, we show two important m... |

13 |
Multiplicative denoising and deblurring: Theory and algorithms
- Rudin, Lions, et al.
- 2003
(Show Context)
Citation Context ...ect to the additional constraints. This is of importance, e.g., for nonnegativity constraints or for multiplicative noise, where one wants to choose � � �2 f H(u, f)= u subject to the constraint (cf. =-=[37]-=-) � f C(u)=−1+ Ω u =0. If the constraint set is not empty, the analysis of well-definedness of the iterates is of similar difficulty as in the unconstrained case, but the convergence analysis cannot b... |

12 |
Analysis of regularized total variation penalty methods for denoising, Inverse Problems
- DOBSON, SCHERZER
- 1996
(Show Context)
Citation Context ...ard Tikhonov-type regularization techniques. • Approximations of total variation: In several instances, one rather minimizes the smooth approximation � � Jɛ(u)= |∇u| 2 + ɛ2 Ω for some ɛ>0 (cf., e.g., =-=[19]-=-). Such an approximation simplifies numerical computations due to the differentiability of Jɛ and may help to avoid the staircasing effect in some cases. The analysis can be carried out in the same wa... |

11 |
O.: Variational methods on the space of functions of bounded Hessian for convexification and denoising
- Hinterberger, Scherzer
- 2006
(Show Context)
Citation Context ...esting, in particular, for deblurring and for more general fitting functionals as outlined below. • Derivatives of bounded variation: Another obvious generalization considered by several authors (cf. =-=[15, 27]-=-) is to use the bounded variation of ∇u, i.e., � J(u)= |D 2 u|, Ω where D2u denotes the Hessian of u, or even more general functionals of the form � J(u)= ϕ(u, ∇u, D 2 u), Ω with convex ϕ : R × R 2 × ... |

10 |
An iterative algorithm for signal reconstruction from bispectrum
- Cetin
- 1991
(Show Context)
Citation Context ...timate above is satisfied. These results are generalized and made precise in section 3. Iterative procedures involving Bregman distance have been used before in signal processing algorithms, e.g., in =-=[9, 10]-=-. There, and in all the other applications that we are aware of, the goal was to accelerate the computation of the solution to a fixed problem, e.g., to solve the ROF minimization equation (1.1). The ... |

10 | Nonstationary iterated Tikhonov regularization. JOTA
- Hanke, Groetsch
- 1998
(Show Context)
Citation Context ...n process. Our analysis below is motivated by that of Hanke [25], who analyzed Levenberg–Marquardt methods for ill-posed problems (also related to nonstationary iterative Tikhonov regularization (cf. =-=[26, 24]-=-) and inverse scale space methods (cf. [40])), which turns out to be a special case of our iterative regularization strategy when using a quadratic regularization functional J(u)=�u�2 for some Hilbert... |

8 |
Simultaneous structure and texture inpainting
- Bertalmio, Vese, et al.
(Show Context)
Citation Context ...y and in [41] using a combination of sparse representations and total variation regularization. One of the many reasons to separate cartoon from texture is to improve image inpainting algorithms. See =-=[5]-=- for a successful approach to this and [4] for a pioneering paper on this subject. We also mention here that using duality, Chambolle [11] constructed an algorithm solving for v directly in a way that... |

7 |
Decomposing an image: Application to textured images and
- Aujol, Aubert, et al.
- 2005
(Show Context)
Citation Context ...does not do as well in separating cartoon from texture as the Vese–Osher model [45]. See also [3] for an explanation of this phenomenon. Additional work on a cartoon/texture decomposition was done in =-=[2]-=- using duality and in [41] using a combination of sparse representations and total variation regularization. One of the many reasons to separate cartoon from texture is to improve image inpainting alg... |

7 |
Reconstruction of signals from Fourier transform samples
- Cetin
- 1989
(Show Context)
Citation Context ...timate above is satisfied. These results are generalized and made precise in section 3. Iterative procedures involving Bregman distance have been used before in signal processing algorithms, e.g., in =-=[9, 10]-=-. There, and in all the other applications that we are aware of, the goal was to accelerate the computation of the solution to a fixed problem, e.g., to solve the ROF minimization equation (1.1). The ... |

5 |
Nonstationary iterated Tikhonov-Morozov method and third order differential equations for the evaluation of unbounded operators
- Groetsch, Scherzer
(Show Context)
Citation Context ...n process. Our analysis below is motivated by that of Hanke [25], who analyzed Levenberg–Marquardt methods for ill-posed problems (also related to nonstationary iterative Tikhonov regularization (cf. =-=[26, 24]-=-) and inverse scale space methods (cf. [40])), which turns out to be a special case of our iterative regularization strategy when using a quadratic regularization functional J(u)=�u�2 for some Hilbert... |

5 |
Estimées localement fortement homogènes
- Nikolova
- 1997
(Show Context)
Citation Context ...iational problem the Rudin–Osher–Fatemi (ROF) model. It has been used and analyzed by several authors in several different contexts (cf. [1, 8, 12, 17, 31, 36, 47]). Also, in [12] and subsequently in =-=[31, 32, 33, 36]-=- the “staircasing” effect of this model was analyzed. No completely satisfying remedy has yet been found; e.g., see our results in Figures 1(c)–1(f) in this work for 0 ≤ x ≤ 120. In spite of this phen... |

3 |
An algorithm for total variation regularization and denoising
- Chambolle
(Show Context)
Citation Context ...on from texture is to improve image inpainting algorithms. See [5] for a successful approach to this and [4] for a pioneering paper on this subject. We also mention here that using duality, Chambolle =-=[11]-=- constructed an algorithm solving for v directly in a way that simplifies the calculations needed to solve (1.1), (1.8), and (1.12). Duality was also used in [14] to solve (1.1). We note that for each... |

1 |
Geometric processing via normal maps
- Tasdizen, Whitaker, et al.
(Show Context)
Citation Context ...eresting approach to blind deconvolution, also minimizing functionals involving the BV seminorm. 2. Using geometry and iterative regularization. Our present work has several immediate antecedents. In =-=[44]-=-, Tasdizen et al. processed deformable surfaces via the level set method [34]. The idea was used to (a) first process the unit normals to a given initial surface and (b) deform the surface so as to si... |