## Filling-in by joint interpolation of vector fields and gray levels (2001)

### Cached

### Download Links

- [www.tecn.upf.es]
- [www.dtic.upf.edu]
- [www.iua.upf.es]
- [www.dtic.upf.edu]
- [iie.fing.edu.uy]
- [iie.fing.edu.uy]
- [ima.umn.edu]
- [www.ceremade.dauphine.fr]
- [www.iua.upf.es]
- DBLP

### Other Repositories/Bibliography

Venue: | IEEE Trans. Image Processing |

Citations: | 129 - 22 self |

### BibTeX

@ARTICLE{Ballester01filling-inby,

author = {Coloma Ballester and M. Bertalmio and Associate Member and Guillermo Sapiro and Joan Verdera},

title = {Filling-in by joint interpolation of vector fields and gray levels},

journal = {IEEE Trans. Image Processing},

year = {2001},

volume = {10},

pages = {1200--1211}

}

### Years of Citing Articles

### OpenURL

### Abstract

Abstract—A variational approach for filling-in regions of missing data in digital images is introduced in this paper. The approach is based on joint interpolation of the image gray-levels and gradient/isophotes directions, smoothly extending in an automatic fashion the isophote lines into the holes of missing data. This interpolation is computed by solving the variational problem via its gradient descent flow, which leads to a set of coupled second order partial differential equations, one for the gray-levels and one for the gradient orientations. The process underlying this approach can be considered as an interpretation of the Gestaltist’s principle of good continuation. No limitations are imposed on the topology of the holes, and all regions of missing data can be simultaneously processed, even if they are surrounded by completely different structures. Applications of this technique include the restoration of old photographs and removal of superimposed text like dates, subtitles, or publicity. Examples of these applications are given. We conclude the paper with a number of theoretical results on the proposed variational approach and its corresponding gradient descent flow. Index Terms—Filling-in, Gestalt principles, image gradients, image gray-levels, interpolation, partial differential equations, variational approach. I.

### Citations

1562 |
Nonlinear total variation based noise removal algorithms
- Rudin, Osher, et al.
- 1992
(Show Context)
Citation Context ...ors present a clear and intuitive axiomatic approach to the problem. The main algorithm they propose after an interesting discussion of the inpainting problem is to minimize the total variation (TV), =-=[35]-=-, of the image inside the hole (they also use, as proposed in [6], [7] and further studied in this paper, a band surrounding the region). They address in addition the interpolation and filling-in in t... |

829 | Texture synthesis by nonparametric sampling
- Efros, Leung
- 1999
(Show Context)
Citation Context .... The basic idea here is to select a texture and synthesize it inside the region to be filled-in (the hole). Although outstanding texture synthesis results have been reported in the literature, e.g., =-=[14]-=-, [19], [22], and [36], these algorithms require the user to select the texture to be copied into the hole. For images in which the region to be replaced covers several different structures, the user ... |

654 |
Functions of bounded variation and free discontinuity problems
- Ambrosio, Fusco, et al.
- 2000
(Show Context)
Citation Context ...ded to all ) as the Radon-Nikodym derivative of the measure with respect to , i.e., it formally satisfies and, also, a.e. For further information concerning functions of bounded variation we refer to =-=[1]-=-, [17], and [40]. Let us now introduce the function spaces for . Let be an open bounded subset of with a Lipschitz boundary. We define and is a Radon measure in The Trace Theorem [2], [10] guarantees ... |

509 |
Convex analysis and variational problems
- Ekeland, Temam
- 1999
(Show Context)
Citation Context ...al in the definition of is well defined. The first integral requires the use of the space of bounded variation functions. Thus our admissible class is a.e. . We propose Minimize (21) As is well-known =-=[15]-=-, [21], the solution of this problem has to be understood in a weak sense as the solution of the problem Minimize (22) Then we have the following result. Theorem 1: Let , with . Then there is a functi... |

407 | Image inpainting
- Bertalmio, Sapiro, et al.
- 2000
(Show Context)
Citation Context ...etouching damaged paintings and photographs. The basic idea is to fill-in the gap of missing data in a form that it is nondetectable by an ordinary observer. In art, this process is called inpainting =-=[6]-=-, [7], [16], [25], [39]. Since the early days of art and photography, filling-in and inpainting has been done by professional artists. Imitating their performance with semi-automatic digital technique... |

359 | The Physics of Liquid Crystals - Gennes, Prost - 1993 |

311 | An introduction to the mathematical theory of the Navier-Stokes equations, Vol - Galdi - 1994 |

252 |
Ordinary differential equations, transport theory and Sobolev spaces
- DiPerna, Lions
- 1989
(Show Context)
Citation Context ...ess of solutions of (18) is guaranteed when is a Lipschitz vector field, a very strong regularity assumption, which excludes any singularity for . More general existence results have been obtained in =-=[13]-=- via the study of transport equations, indeed, via formulations analog to (19). Typically, they are assuming that is in some Sobolev space like , with some other integrability assumptions, and . These... |

233 |
Analyse mathématique et calcul numérique pour les sciences techniques
- Dautray, Lions
- 1988
(Show Context)
Citation Context ...ded subset of IR2 with a Lipschitz boundary. We define W 1;p(div; Q) = f` 2 Lp(Q)2 : div(`) 2 Lp(Q)g; 1 ^ p ! 1; and M (div; Q) = f` 2 L1(Q)2 : div(`) is a Radon measure in Qg: The Trace Theorem ([2],=-=[10]-=-) guarantees that the normal component ` \Deltasnj@Q, is well defined for vector fields ` in W 1;p(div; Q), or in M (div; Q). To simplify our notation we shall assume that W 1;1(div; Q) represents the... |

137 |
R.F.: Measure Theory and Fine
- Evans, Gariepy
- 1992
(Show Context)
Citation Context ...o all ) as the Radon-Nikodym derivative of the measure with respect to , i.e., it formally satisfies and, also, a.e. For further information concerning functions of bounded variation we refer to [1], =-=[17]-=-, and [40]. Let us now introduce the function spaces for . Let be an open bounded subset of with a Lipschitz boundary. We define and is a Radon measure in The Trace Theorem [2], [10] guarantees that t... |

108 | Texture characterization via joint statistics of wavelet coefficient magnitudes
- Simoncelli, Portilla
- 1998
(Show Context)
Citation Context ...is to select a texture and synthesize it inside the region to be filled-in (the hole). Although outstanding texture synthesis results have been reported in the literature, e.g., [14], [19], [22], and =-=[36]-=-, these algorithms require the user to select the texture to be copied into the hole. For images in which the region to be replaced covers several different structures, the user would need to go throu... |

93 | Level lines based disocclusion
- Masnou, Morel
- 1998
(Show Context)
Citation Context ...tions at the same gray level. (Other researchers, e.g., Jacobs et al. have followed this interesting research area, mainly developing techniques for smooth curve continuation.) Masnou and Morel [31], =-=[32]-=- recently extended these ideas, presenting a very elegant and inspiring formal variational formulation for disocclusion and a particular practical algorithm implementing some of the ideas in this form... |

85 |
Interpolation of missing data in image sequences
- Kokaram, Morris, et al.
- 1995
(Show Context)
Citation Context ...ota, Minneapolis, MN 55455, USA (e-mail: guille@ece.umn.edu). Publisher Item Identifier S 1057-7149(01)06032-8. 1057–7149/01$10.00 © 2001 IEEE an active area of research; see, for example, [8], [23], =-=[27]-=-, and [28], and the papers discussed in this paper. The goal of this work is to introduce a novel algorithm for automatically filling-in gaps in the image. In this paper, we follow the suggestions in ... |

63 |
Paring between measures and bounded functions and compensated compactness
- ANZELLOTTI
- 1983
(Show Context)
Citation Context ...ation we refer to [1], [17], and [40]. Let us now introduce the function spaces for . Let be an open bounded subset of with a Lipschitz boundary. We define and is a Radon measure in The Trace Theorem =-=[2]-=-, [10] guarantees that the normal component , is well defined for vector fields in , or in . To simplify our notation we shall assume that represents the space . Next, we shall give a sense to the int... |

56 | Minimizing total variation flow
- Andreu, Ballester, et al.
(Show Context)
Citation Context ...ge U0(t; x) obtained by regularizing u0(x), x 2 B, with the equation @u @t = div ` ru jruj ' in Q = (0; 1) \ThetasB @u @j = 0 in S = (0; 1) \Thetas@B u(0; x) = u0(x) for x 2 B: (16) As it is shown in =-=[3]-=-, this equation permits a regularization of the vector field of directions of the gradient of u, i.e., there is a vector field z, jzj ^ 1, such that ut = div(z) and R B z \Deltasru = R B jruj. Moreove... |

48 |
Pyramid based texture analysis/synthesis
- Heeger, Bergen
- 1995
(Show Context)
Citation Context ...basic idea here is to select a texture and synthesize it inside the region to be filled-in (the hole). Although outstanding texture synthesis results have been reported in the literature, e.g., [14], =-=[19]-=-, [22], and [36], these algorithms require the user to select the texture to be copied into the hole. For images in which the region to be replaced covers several different structures, the user would ... |

48 |
R: Spatial and temporal properties of illusory contours and amodal boundary completion. Vision Res
- Ringach, Shapley
- 1996
(Show Context)
Citation Context ...the angle of arrival of the level lines at the gap, and pursuing a smooth interpolation of it as done here, it is also supported by research in perception, from the Gestalt to more recent work, e.g., =-=[34]-=-. To conclude this section, the interested reader is referred to the works of Nitzberg–Mumford–Shiota, Masnou–Morel, and Chan–Shen (as well as our previous work) to study other interesting and very re... |

47 | Combining Frequency and spatial domain information for fast interactive image noise removal. Computer Graphics
- Hirani, Totsuka
- 1996
(Show Context)
Citation Context ...idea here is to select a texture and synthesize it inside the region to be filled-in (the hole). Although outstanding texture synthesis results have been reported in the literature, e.g., [14], [19], =-=[22]-=-, and [36], these algorithms require the user to select the texture to be copied into the hole. For images in which the region to be replaced covers several different structures, the user would need t... |

36 |
differentiable functions. Graduate Texts in Mathematics 120
- Ziemer, Weakly
- 1989
(Show Context)
Citation Context ... the Radon-Nikodym derivative of the measure with respect to , i.e., it formally satisfies and, also, a.e. For further information concerning functions of bounded variation we refer to [1], [17], and =-=[40]-=-. Let us now introduce the function spaces for . Let be an open bounded subset of with a Lipschitz boundary. We define and is a Radon measure in The Trace Theorem [2], [10] guarantees that the normal ... |

34 |
Mathematical models for local deterministic inpaintings
- Chan, Shen
- 2000
(Show Context)
Citation Context ...ar model in order to be able to define a formal variational approach to the filling-in problem. The work in [6] and [7] inspired a very elegant approach to the filling-in problem recently reported in =-=[9]-=- (this work was performed independently to the one reported in this paper). 2 The authors present a clear and intuitive axiomatic approach to the problem. The main algorithm they propose after an inte... |

28 | The Dirichlet problem for the total variation flow
- ANDREU, BALLESTER, et al.
- 2001
(Show Context)
Citation Context ...richlet boundary data and initial condition constructed as an ad-hoc interpolation that will be corrected by the equation. General existence results which can be adapted to this case can be found in (=-=[4]-=-). Note that the vector field ` is assumed to be known in this case. This limits the usefulness of this model. But we present some experiments below to illustrate the role of this term. To minimize th... |

27 | Detection and removal of line scratches in motion picture films
- Joyeux, Buisson, et al.
- 1999
(Show Context)
Citation Context ...Minnesota, Minneapolis, MN 55455, USA (e-mail: guille@ece.umn.edu). Publisher Item Identifier S 1057-7149(01)06032-8. 1057–7149/01$10.00 © 2001 IEEE an active area of research; see, for example, [8], =-=[23]-=-, [27], and [28], and the papers discussed in this paper. The goal of this work is to introduce a novel algorithm for automatically filling-in gaps in the image. In this paper, we follow the suggestio... |

27 |
R.: Dual spaces of stresses and strains, with applications to Hencky plasticity
- Kohn, Temam
(Show Context)
Citation Context ...ll give a sense to the integrals of bounded vector fields with divergence in integrated with respect to the gradient of a BV function. For that, we shall need some results from [2] (see also [11] and =-=[26]-=-). Let be an open bounded subset of with Lipschitz continuous boundary. Let and be such that . Following [2], let (4) (5)sBALLESTER et al.: FILLING-IN BY JOINT INTERPOLATION OF VECTOR FIELDS AND GRAY ... |

27 |
Sur les équations différentielles ordinaires et les équations de transport. (French) [On ordinary differential equations and transport equations
- Lions
- 1998
(Show Context)
Citation Context ..., via formulations analog to (19). Typically, they are assuming that is in some Sobolev space like , with some other integrability assumptions, and . These results have been further extended in [12], =-=[30]-=-. In particular, Lions in [30] proves a.e. existence of solutions of (18) for vector fields which are piecewise in in a precise sense defined by the author. As observed in these papers, it is not know... |

23 |
Semicontinuity and relaxation properties of curvature depending functional in 2D. Annali della Scuola Normale di Pisa, Classe di Scienze, 4e série
- Bellettini, Maso, et al.
- 1993
(Show Context)
Citation Context ...g occlusions with the goal of image segmentation, since this criterion yields smooth, short, and not too curvy curves. In terms of characteristic functions, Euler’s elastica can be written as (13) In =-=[5]-=-, it was shown that this functional is not lower semicontinuous. The functional proposed by Masnou and Morel [31], [32] can be interpreted as a relaxation of it, since it integrates functionals like t... |

20 | Exact solutions to total variation regularization problems
- Strong, Chan
- 1996
(Show Context)
Citation Context ...not have a rigorous proof that this is so. The functional was introduced on a heuristic basis, but relaxations may occur as they occur in (16), where may represent when computed on a flat region [3], =-=[37]-=-. This requires further study and we shall pursue it elsewhere. 6) The choice made in Remark 5 of decomposing the image into upper level sets, interpolating them and reconstructing the function , intr... |

18 |
Filtrage et Desocclusion d’Images par Méthodes d’Ensembles de Niveau, Thèse, Univ. Paris-Dauphine
- Masnou
- 1998
(Show Context)
Citation Context ...T-junctions at the same gray level. (Other researchers, e.g., Jacobs et al. have followed this interesting research area, mainly developing techniques for smooth curve continuation.) Masnou and Morel =-=[31]-=-, [32] recently extended these ideas, presenting a very elegant and inspiring formal variational formulation for disocclusion and a particular practical algorithm implementing some of the ideas in thi... |

17 |
The Restorer’s Handbook of Easel Painting
- Emile-Male
- 1976
(Show Context)
Citation Context ...damaged paintings and photographs. The basic idea is to fill-in the gap of missing data in a form that it is nondetectable by an ordinary observer. In art, this process is called inpainting [6], [7], =-=[16]-=-, [25], [39]. Since the early days of art and photography, filling-in and inpainting has been done by professional artists. Imitating their performance with semi-automatic digital techniques is curren... |

17 |
Functionals with linear growth in the calculus of variations
- Giaquinta, Modica, et al.
- 1979
(Show Context)
Citation Context ...the definition of is well defined. The first integral requires the use of the space of bounded variation functions. Thus our admissible class is a.e. . We propose Minimize (21) As is well-known [15], =-=[21]-=-, the solution of this problem has to be understood in a weak sense as the solution of the problem Minimize (22) Then we have the following result. Theorem 1: Let , with . Then there is a function suc... |

14 | Pseudosolutions of the time dependent minimal surface problem - Lichnewski, Temam - 1978 |

14 |
The Ravished Image
- Walden
- 1985
(Show Context)
Citation Context ...tings and photographs. The basic idea is to fill-in the gap of missing data in a form that it is nondetectable by an ordinary observer. In art, this process is called inpainting [6], [7], [16], [25], =-=[39]-=-. Since the early days of art and photography, filling-in and inpainting has been done by professional artists. Imitating their performance with semi-automatic digital techniques is currently Manuscri... |

13 |
The Commissar Vanishes
- King
(Show Context)
Citation Context ...d paintings and photographs. The basic idea is to fill-in the gap of missing data in a form that it is nondetectable by an ordinary observer. In art, this process is called inpainting [6], [7], [16], =-=[25]-=-, [39]. Since the early days of art and photography, filling-in and inpainting has been done by professional artists. Imitating their performance with semi-automatic digital techniques is currently Ma... |

12 |
A few remarks on ordinary differential equations
- Desjardins
- 1996
(Show Context)
Citation Context ...indeed, via formulations analog to (19). Typically, they are assuming that is in some Sobolev space like , with some other integrability assumptions, and . These results have been further extended in =-=[12]-=-, [30]. In particular, Lions in [30] proves a.e. existence of solutions of (18) for vector fields which are piecewise in in a precise sense defined by the author. As observed in these papers, it is no... |

11 | Ordinary dierential equations, transport theory and Sobolev spaces - DiPerna, Lions - 1989 |

10 |
A New Image Processing Primitive: Reconstructing Images From Modified Flow Fields
- Kenney, Langan
- 1999
(Show Context)
Citation Context ...3 Interpolating gray values along the integral curves of a vector field Our purpose in this section is to further discuss the term Z ~\Omegasjruj \Gammas` \Deltasru: (17) in functional (10) (see also =-=[24]-=- for a related, L2 and Poisson-equation based, approach of gray value reconstruction from image gradients). We shall see that (when ` is known), when minimizing (17), we are constructing the function ... |

10 | Photoshop retouching handbook - Braverman - 1998 |

9 |
Minimizing total variation
- Andreu, Ballester, et al.
- 2001
(Show Context)
Citation Context ...alize the algorithm of steepest descent, a regularization of outside may be constructed as the vector field of directions of the image obtained by regularizing , with in in for (16) As it is shown in =-=[3]-=-, this equation permits a regularization of the vector field of directions of the gradient of , i.e., there is a vector field , such that and . Moreover, for each if for all . In this way, we initiali... |

8 | On the continuity of the trace of vector functions with bounded deformation - Temam - 1981 |

5 |
Analyse mathematique et calcul numerique pour les sciences et les techniques
- DAUTRAY, LIONS
- 1985
(Show Context)
Citation Context ... we refer to [1], [17], and [40]. Let us now introduce the function spaces for . Let be an open bounded subset of with a Lipschitz boundary. We define and is a Radon measure in The Trace Theorem [2], =-=[10]-=- guarantees that the normal component , is well defined for vector fields in , or in . To simplify our notation we shall assume that represents the space . Next, we shall give a sense to the integrals... |

4 |
Photoshop Retouching Handbook, IDG Books Worldwide
- Braverman
- 1998
(Show Context)
Citation Context ...otography, filling-in and inpainting has been done by professional artists. Imitating their performance with semi-automatic digital techniques is currently an active area of research, see for example =-=[23, 27, 28, 8]-=- and the papers discussed below. The goal of this work is to introduce a novel algorithm for automatically filling-in gaps in the image. In this article we follow the suggestions in the conclusions se... |

2 |
Introduction aux 'Equations aux Deriv'ees Partielles Elliptiques, Diderot Ed
- Demengel
- 1999
(Show Context)
Citation Context ...t, we shall give a sense to the integrals of bounded vector fields with divergence in integrated with respect to the gradient of a BV function. For that, we shall need some results from [2] (see also =-=[11]-=- and [26]). Let be an open bounded subset of with Lipschitz continuous boundary. Let and be such that . Following [2], let (4) (5)sBALLESTER et al.: FILLING-IN BY JOINT INTERPOLATION OF VECTOR FIELDS ... |

2 |
The Commissar Vanishes. Henry Holt and Company
- King
- 1997
(Show Context)
Citation Context ...touching damaged paintings and photographs. The basic idea is to fill-in the gap of missing data in a form that it is non-detectable by an ordinary observer. In art, this process is called inpainting =-=[39, 16, 25, 6, 7]-=-. Since the early days of art and photography, filling-in and inpainting has been done by professional artists. Imitating their performance with semi-automatic digital techniques is currently an activ... |

2 | A few remarks on ordinary dierential equations - Desjardins - 1996 |

1 | The Dirichlet problem for the Total Variation preprint - Andreu, Ballester, et al. |

1 | Sur les equations dierentielles ordinaires et les equations de transport - Lions - 1998 |