Results 1  10
of
425
Fields of experts: A framework for learning image priors
 In CVPR
, 2005
"... We develop a framework for learning generic, expressive image priors that capture the statistics of natural scenes and can be used for a variety of machine vision tasks. The approach extends traditional Markov Random Field (MRF) models by learning potential functions over extended pixel neighborhood ..."
Abstract

Cited by 237 (4 self)
 Add to MetaCart
(Show Context)
We develop a framework for learning generic, expressive image priors that capture the statistics of natural scenes and can be used for a variety of machine vision tasks. The approach extends traditional Markov Random Field (MRF) models by learning potential functions over extended pixel neighborhoods. Field potentials are modeled using a ProductsofExperts framework that exploits nonlinear functions of many linear filter responses. In contrast to previous MRF approaches all parameters, including the linear filters themselves, are learned from training data. We demonstrate the capabilities of this Field of Experts model with two example applications, image denoising and image inpainting, which are implemented using a simple, approximate inference scheme. While the model is trained on a generic image database and is not tuned toward a specific application, we obtain results that compete with and even outperform specialized techniques. 1.
Region Filling and Object Removal by ExemplarBased Image Inpainting
, 2004
"... A new algorithm is proposed for removing large objects from digital images. The challenge is to fill in the hole that is left behind in a visually plausible way. In the past, this problem has been addressed by two classes of algorithms: 1) “texture synthesis” algorithms for generating large image re ..."
Abstract

Cited by 190 (1 self)
 Add to MetaCart
(Show Context)
A new algorithm is proposed for removing large objects from digital images. The challenge is to fill in the hole that is left behind in a visually plausible way. In the past, this problem has been addressed by two classes of algorithms: 1) “texture synthesis” algorithms for generating large image regions from sample textures and 2) “inpainting ” techniques for filling in small image gaps. The former has been demonstrated for “textures”—repeating twodimensional patterns with some stochasticity; the latter focus on linear “structures ” which can be thought of as onedimensional patterns, such as lines and object contours. This paper presents a novel and efficient algorithm that combines the advantages of these two approaches. We first note that exemplarbased texture synthesis contains the essential process required to replicate both texture and structure; the success of structure propagation, however, is highly dependent on the order in which the filling proceeds. We propose a bestfirst algorithm in which the confidence in the synthesized pixel values is propagated in a manner similar to the propagation of information in inpainting. The actual color values are computed using exemplarbased synthesis. In this paper, the simultaneous propagation of texture and structure information is achieved by a single, efficient algorithm. Computational efficiency is achieved by a blockbased sampling process. A number of examples on real and synthetic images demonstrate the effectiveness of our algorithm in removing large occluding objects, as well as thin scratches. Robustness with respect to the shape of the manually selected target region is also demonstrated. Our results compare favorably to those obtained by existing techniques.
Seam carving for contentaware image resizing
 ACM Trans. Graph
, 2007
"... Figure 1: A seam is a connected path of low energy pixels in an image. On the left is the original image with one horizontal and one vertical seam. In the middle the energy function used in this example is shown (the magnitude of the gradient), along with the vertical and horizontal path maps used t ..."
Abstract

Cited by 178 (7 self)
 Add to MetaCart
Figure 1: A seam is a connected path of low energy pixels in an image. On the left is the original image with one horizontal and one vertical seam. In the middle the energy function used in this example is shown (the magnitude of the gradient), along with the vertical and horizontal path maps used to calculate the seams. By automatically carving out seams to reduce image size, and inserting seams to extend it, we achieve contentaware resizing. The example on the top right shows our result of extending in one dimension and reducing in the other, compared to standard scaling on the bottom right. Effective resizing of images should not only use geometric constraints, but consider the image content as well. We present a simple image operator called seam carving that supports contentaware image resizing for both reduction and expansion. A seam is an optimal 8connected path of pixels on a single image from top to bottom, or left to right, where optimality is defined by an image energy function. By repeatedly carving out or inserting seams in one direction we can change the aspect ratio of an image. By applying these operators in both directions we can retarget the image to a new size. The selection and order of seams protect the content of the image, as defined by the energy function. Seam carving can also be used for image content enhancement and object removal. We support various visual saliency measures for defining the energy of an image, and can also include user input to guide the process. By storing the order of seams in an image we create multisize images, that are able to continuously change in real time to fit a given size.
Simultaneous Structure and Texture Image Inpainting
, 2003
"... An algorithm for the simultaneous fillingin of texture and structure in regions of missing image information is presented in this paper. The basic idea is to first decompose the image into the sum of two functions with different basic characteristics, and then reconstruct each one of these function ..."
Abstract

Cited by 162 (12 self)
 Add to MetaCart
(Show Context)
An algorithm for the simultaneous fillingin of texture and structure in regions of missing image information is presented in this paper. The basic idea is to first decompose the image into the sum of two functions with different basic characteristics, and then reconstruct each one of these functions separately with structure and texture fillingin algorithms. The first function used in the decomposition is of bounded variation, representing the underlying image structure, while the second function captures the texture and possible noise. The region of missing information in the bounded variation image is reconstructed using image inpainting algorithms, while the same region in the texture image is filledin with texture synthesis techniques. The original image is then reconstructed adding back these two subimages. The novel contribution of this paper is then in the combination of these three previously developed components, image decomposition with inpainting and texture synthesis, which permits the simultaneous use of fillingin algorithms that are suited for different image characteristics. Examples on real images show the advantages of this proposed approach.
Mathematical Models for Local Nontexture Inpaintings
 SIAM J. Appl. Math
, 2002
"... Inspired by the recent work of Bertalmio et al. on digital inpaintings [SIGGRAPH 2000], we develop general mathematical models for local inpaintings of nontexture images. On smooth regions, inpaintings are connected to the harmonic and biharmonic extensions, and inpainting orders are analyzed. For i ..."
Abstract

Cited by 162 (30 self)
 Add to MetaCart
(Show Context)
Inspired by the recent work of Bertalmio et al. on digital inpaintings [SIGGRAPH 2000], we develop general mathematical models for local inpaintings of nontexture images. On smooth regions, inpaintings are connected to the harmonic and biharmonic extensions, and inpainting orders are analyzed. For inpaintings involving the recovery of edges, we study a variational model that is closely connected to the classical total variation (TV) denoising model of Rudin, Osher, and Fatemi [PhSG D, 60 (1992), pp. 259268]. Other models are also discussed based on the MumfordShah regularity [Comm. Pure Appl. Mathq XLII (1989), pp. 577685] and curvature driven di#usions (CDD) of Chan and Shen [J. Visual Comm. Image Rep., 12 (2001)]. The broad applications of the inpainting models are demonstrated through restoring scratched old photos, disocclusion in vision analysis, text removal, digital zooming, and edgebased image coding.
Fillingin by joint interpolation of vector fields and gray levels
 IEEE Trans. Image Processing
, 2001
"... Abstract—A variational approach for fillingin regions of missing data in digital images is introduced in this paper. The approach is based on joint interpolation of the image graylevels and gradient/isophotes directions, smoothly extending in an automatic fashion the isophote lines into the holes ..."
Abstract

Cited by 128 (22 self)
 Add to MetaCart
(Show Context)
Abstract—A variational approach for fillingin regions of missing data in digital images is introduced in this paper. The approach is based on joint interpolation of the image graylevels and gradient/isophotes directions, smoothly extending in an automatic fashion the isophote lines into the holes of missing data. This interpolation is computed by solving the variational problem via its gradient descent flow, which leads to a set of coupled second order partial differential equations, one for the graylevels and one for the gradient orientations. The process underlying this approach can be considered as an interpretation of the Gestaltist’s principle of good continuation. No limitations are imposed on the topology of the holes, and all regions of missing data can be simultaneously processed, even if they are surrounded by completely different structures. Applications of this technique include the restoration of old photographs and removal of superimposed text like dates, subtitles, or publicity. Examples of these applications are given. We conclude the paper with a number of theoretical results on the proposed variational approach and its corresponding gradient descent flow. Index Terms—Fillingin, Gestalt principles, image gradients, image graylevels, interpolation, partial differential equations, variational approach. I.
VectorValued Image Regularization with PDEs: A Common Framework for Different Applications
 IEEE Transactions on Pattern Analysis and Machine Intelligence
, 2003
"... We address the problem of vectorvalued image regularization with variational methods and PDE's. From the study of existing formalisms, we propose a unifying framework based on a very local interpretation of the regularization processes. The resulting equations are then specialized into new reg ..."
Abstract

Cited by 128 (8 self)
 Add to MetaCart
(Show Context)
We address the problem of vectorvalued image regularization with variational methods and PDE's. From the study of existing formalisms, we propose a unifying framework based on a very local interpretation of the regularization processes. The resulting equations are then specialized into new regularization PDE's and corresponding numerical schemes that respect the local geometry of vectorvalued images. They are finally applied on a wide variety of image processing problems, including color image restoration, inpainting, magnification and flow visualization.
Sparse representation for color image restoration
 the IEEE Trans. on Image Processing
, 2007
"... Sparse representations of signals have drawn considerable interest in recent years. The assumption that natural signals, such as images, admit a sparse decomposition over a redundant dictionary leads to efficient algorithms for handling such sources of data. In particular, the design of well adapted ..."
Abstract

Cited by 118 (27 self)
 Add to MetaCart
(Show Context)
Sparse representations of signals have drawn considerable interest in recent years. The assumption that natural signals, such as images, admit a sparse decomposition over a redundant dictionary leads to efficient algorithms for handling such sources of data. In particular, the design of well adapted dictionaries for images has been a major challenge. The KSVD has been recently proposed for this task [1], and shown to perform very well for various grayscale image processing tasks. In this paper we address the problem of learning dictionaries for color images and extend the KSVDbased grayscale image denoising algorithm that appears in [2]. This work puts forward ways for handling nonhomogeneous noise and missing information, paving the way to stateoftheart results in applications such as color image denoising, demosaicing, and inpainting, as demonstrated in this paper. EDICS Category: COLCOLR (Color processing) I.
Spacetime video completion
 in Proceedinggs of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’04
, 2004
"... We present a method for spacetime completion of large spacetime “holes ” in video sequences of complex dynamic scenes. The missing portions are filledin by sampling spatiotemporal patches from the available parts of the video, while enforcing global spatiotemporal consistency between all patc ..."
Abstract

Cited by 108 (4 self)
 Add to MetaCart
(Show Context)
We present a method for spacetime completion of large spacetime “holes ” in video sequences of complex dynamic scenes. The missing portions are filledin by sampling spatiotemporal patches from the available parts of the video, while enforcing global spatiotemporal consistency between all patches in and around the hole. This is obtained by posing the task of video completion and synthesis as a global optimization problem with a welldefined objective function. The consistent completion of static scene parts simultaneously with dynamic behaviors leads to realistic looking video sequences. Spacetime video completion is useful for a variety of tasks, including, but not limited to: (i) Sophisticated video removal (of undesired static or dynamic objects) by completing the appropriate static or dynamic background information, (ii) Correction of missing/corrupted video frames in old movies, and (iii) Synthesis of new video frames to add a visual story, modify it, or generate a new one. Some examples of these are shown in the paper. 1.
An iterative regularization method for total variationbased image restoration, Multiscale Model
 Simul
"... Abstract. We introduce a new iterative regularization procedure for inverse problems based on the use of Bregman distances, with particular focus on problems arising in image processing. We are motivated by the problem of restoring noisy and blurry images via variational methods by using total varia ..."
Abstract

Cited by 101 (20 self)
 Add to MetaCart
(Show Context)
Abstract. We introduce a new iterative regularization procedure for inverse problems based on the use of Bregman distances, with particular focus on problems arising in image processing. We are motivated by the problem of restoring noisy and blurry images via variational methods by using total variation regularization. We obtain rigorous convergence results and effective stopping criteria for the general procedure. The numerical results for denoising appear to give significant improvement over standard models, and preliminary results for deblurring/denoising are very encouraging.