Results 1  10
of
33
Scene completion using millions of photographs
 ACM Transactions on Graphics (SIGGRAPH
, 2007
"... Figure 1: Given an input image with a missing region, we use matching scenes from a large collection of photographs to complete the image. What can you do with a million images? In this paper we present a new image completion algorithm powered by a huge database of photographs gathered from the Web. ..."
Abstract

Cited by 188 (10 self)
 Add to MetaCart
Figure 1: Given an input image with a missing region, we use matching scenes from a large collection of photographs to complete the image. What can you do with a million images? In this paper we present a new image completion algorithm powered by a huge database of photographs gathered from the Web. The algorithm patches up holes in images by finding similar image regions in the database that are not only seamless but also semantically valid. Our chief insight is that while the space of images is effectively infinite, the space of semantically differentiable scenes is actually not that large. For many image completion tasks we are able to find similar scenes which contain image fragments that will convincingly complete the image. Our algorithm is entirely datadriven, requiring no annotations or labelling by the user. Unlike existing image completion methods, our algorithm can generate a diverse set of results for each input image and we allow users to select among them. We demonstrate the superiority of our algorithm over existing image completion approaches.
Performance vs Computational Efficiency for Optimizing Single and Dynamic MRFs: Setting the State of the Art with Primal Dual Strategies
"... In this paper we introduce a novel method to address minimization of static and dynamic MRFs. Our approach is based on principles from linear programming and, in particular, on primal dual strategies. It generalizes prior stateoftheart methods such as αexpansion, while it can also be used for ef ..."
Abstract

Cited by 45 (18 self)
 Add to MetaCart
In this paper we introduce a novel method to address minimization of static and dynamic MRFs. Our approach is based on principles from linear programming and, in particular, on primal dual strategies. It generalizes prior stateoftheart methods such as αexpansion, while it can also be used for efficiently minimizing NPhard problems with complex pairwise potential functions. Furthermore, it offers a substantial speedup of a magnitude ten over existing techniques, due to the fact that it exploits information coming not only from the original MRF problem, but also from a dual one. The proposed technique consists of recovering pair of solutions for the primal and the dual such that the gap between them is minimized. Therefore, it can also boost performance of dynamic MRFs, where one should expect that the new new pair of primaldual solutions is closed to the previous one. Promising results in a number of applications, and theoretical, as well as numerical comparisons with the state of the art demonstrate the extreme potentials of this approach.
MRF energy minimization and beyond via dual decomposition
 IN: IEEE PAMI. (2011
"... This paper introduces a new rigorous theoretical framework to address discrete MRFbased optimization in computer vision. Such a framework exploits the powerful technique of Dual Decomposition. It is based on a projected subgradient scheme that attempts to solve an MRF optimization problem by first ..."
Abstract

Cited by 42 (4 self)
 Add to MetaCart
(Show Context)
This paper introduces a new rigorous theoretical framework to address discrete MRFbased optimization in computer vision. Such a framework exploits the powerful technique of Dual Decomposition. It is based on a projected subgradient scheme that attempts to solve an MRF optimization problem by first decomposing it into a set of appropriately chosen subproblems and then combining their solutions in a principled way. In order to determine the limits of this method, we analyze the conditions that these subproblems have to satisfy and we demonstrate the extreme generality and flexibility of such an approach. We thus show that, by appropriately choosing what subproblems to use, one can design novel and very powerful MRF optimization algorithms. For instance, in this manner we are able to derive algorithms that: 1) generalize and extend stateoftheart messagepassing methods, 2) optimize very tight LPrelaxations to MRF optimization, 3) and take full advantage of the special structure that may exist in particular MRFs, allowing the use of efficient inference techniques such as, e.g, graphcut based methods. Theoretical analysis on the bounds related with the different algorithms derived from our framework and experimental results/comparisons using synthetic and real data for a variety of tasks in computer vision demonstrate the extreme potentials of our approach.
Image Completion Using Efficient Belief Propagation via Priority Scheduling and Dynamic Pruning
"... In this paper, a new exemplarbased framework is presented, which treats image completion, texture synthesis and image inpainting in a unified manner. In order to be able to avoid the occurrence of visually inconsistent results, we pose all of the above imageediting tasks in the form of a discrete ..."
Abstract

Cited by 26 (0 self)
 Add to MetaCart
(Show Context)
In this paper, a new exemplarbased framework is presented, which treats image completion, texture synthesis and image inpainting in a unified manner. In order to be able to avoid the occurrence of visually inconsistent results, we pose all of the above imageediting tasks in the form of a discrete global optimization problem. The objective function of this problem is always welldefined, and corresponds to the energy of a discrete Markov Random Field (MRF). For efficiently optimizing this MRF, a novel optimization scheme, called PriorityBP, is then proposed, which carries two very important extensions over the standard Belief Propagation (BP) algorithm: “prioritybased message scheduling ” and “dynamic label pruning”. These two extensions work in cooperation to deal with the intolerable computational cost of BP, which is caused by the huge number of labels associated with our MRF. Moreover, both of our extensions are generic, since they do not rely on the use of domainspecific prior knowledge. They can therefore be applied to any MRF, i.e to a very wide class of problems in image processing and computer vision, thus managing to resolve what is currently considered as one major limitation of the Belief Propagation algorithm: its inefficiency in handling MRFs with very large discrete statespaces. Experimental results on a wide variety of input images are presented, which demonstrate the effectiveness of our imagecompletion framework for tasks such as object removal, texture synthesis, text removal and image inpainting.
Tensor Completion for Estimating Missing Values in Visual Data
"... In this paper we propose an algorithm to estimate missing values in tensors of visual data. The values can be missing due to problems in the acquisition process, or because the user manually identified unwanted outliers. Our algorithm works even with a small amount of samples and it can propagate st ..."
Abstract

Cited by 25 (3 self)
 Add to MetaCart
(Show Context)
In this paper we propose an algorithm to estimate missing values in tensors of visual data. The values can be missing due to problems in the acquisition process, or because the user manually identified unwanted outliers. Our algorithm works even with a small amount of samples and it can propagate structure to fill larger missing regions. Our methodology is built on recent studies about matrix completion using the matrix trace norm. The contribution of our paper is to extend the matrix case to the tensor case by laying out the theoretical foundations and then by building a working algorithm. First, we propose a definition for the tensor trace norm, that generalizes the established definition of the matrix trace norm. Second, similar to matrix completion, the tensor completion is formulated as a convex optimization problem. Unfortunately, the straightforward problem extension is significantly harder to solve than the matrix case because of the dependency among multiple constraints. To tackle this problem, we employ a relaxation technique to separate the dependant relationships and use the block coordinate descent (BCD) method to achieve a globally optimal solution. Our experiments show potential applications of our algorithm and the quantitative evaluation indicates that our method is more accurate and robust than heuristic approaches. 1.
Stereoscopic Inpainting: Joint Color and Depth Completion from Stereo Images
"... We present a novel algorithm for simultaneous color and depth inpainting. The algorithm takes stereo images and estimated disparity maps as input and fills in missing color and depth information introduced by occlusions or object removal. We first complete the disparities for the occlusion regions u ..."
Abstract

Cited by 13 (0 self)
 Add to MetaCart
(Show Context)
We present a novel algorithm for simultaneous color and depth inpainting. The algorithm takes stereo images and estimated disparity maps as input and fills in missing color and depth information introduced by occlusions or object removal. We first complete the disparities for the occlusion regions using a segmentationbased approach. The completed disparities can be used to facilitate the user in labeling objects to be removed. Since part of the removed regions in one image is visible in the other, we mutually complete the two images through 3D warping. Finally, we complete the remaining unknown regions using a depthassisted texture synthesis technique, which simultaneously fills in both color and depth. We demonstrate the effectiveness of the proposed algorithm on several challenging data sets. 1.
Image inpainting considering brightness change and spatial locality of textures
 in Proc. Int. Conf. on Computer Vision Theory and Applications (VISAPP
"... image inpainting, image completion, energy minimization Image inpainting is a tequnique for removing undesired visual objects in images and filling the missing regions with plausible textures. Conventionally, the missing parts of an image are completed by optimizing the objective function, which is ..."
Abstract

Cited by 9 (0 self)
 Add to MetaCart
image inpainting, image completion, energy minimization Image inpainting is a tequnique for removing undesired visual objects in images and filling the missing regions with plausible textures. Conventionally, the missing parts of an image are completed by optimizing the objective function, which is defined based on pattern similarity between the missing region and the rest of the image (data region). However, unnatural textures are easily generated due to two factors: (1) available samples in the data region are quite limited, and (2) pattern similarity is one of the required conditions but is not sufficient for reproducing natural textures. In this paper, in order to improve the image quality of completed texture, the objective function is extended by allowing brightness changes of sample textures (for (1)) and introducing spatial locality as an additional constraint (for (2)). The effectiveness of these extensions is successfully demonstrated by applying the proposed method to one hundred images and comparing the results with those obtained by the conventional methods. 1
A Unifying Framework for Image Inpainting
, 2009
"... Inpainting is the art of modifying an image in a form that is not detectable by an ordinary observer. There are numerous and very different approaches to tackle the inpainting problem, but we point out that the most successful inpainting algorithms are based on one or two of the following three basi ..."
Abstract

Cited by 7 (0 self)
 Add to MetaCart
(Show Context)
Inpainting is the art of modifying an image in a form that is not detectable by an ordinary observer. There are numerous and very different approaches to tackle the inpainting problem, but we point out that the most successful inpainting algorithms are based on one or two of the following three basic techniques: copyandpaste texture synthesis, geometric PDE’s, and coherence among neighboring pixels. We combine these three building blocks in a unifying variational model, and provide a working algorithm for image inpainting trying to approximate the minimum of the proposed energy functional. Our experiments show that the combination of all three terms of the proposed energy works better than taking each term separately, and the results obtained are stateoftheart. Index Terms Image inpainting, variational models, texture synthesis, PDE’s.
Hybrid learning of large jigsaws
"... A jigsaw is a recently proposed generative model that describes an image as a composition of nonoverlapping patches of varying shape, extracted from a latent image. By learning the latent jigsaw image which best explains a set of images, it is possible to discover the shape, size and appearance of ..."
Abstract

Cited by 7 (0 self)
 Add to MetaCart
(Show Context)
A jigsaw is a recently proposed generative model that describes an image as a composition of nonoverlapping patches of varying shape, extracted from a latent image. By learning the latent jigsaw image which best explains a set of images, it is possible to discover the shape, size and appearance of repeated structures in the images. A challenge when learning this model is the very large space of possible jigsaw pixels which can potentially be used to explain each image pixel. The previous method of inference for this model scales linearly with the number of jigsaw pixels, making it unusable for learning the large jigsaws needed for many practical applications. In this paper, we make three contributions that enable the learning of large jigsaws a novel sparse belief propagation algorithm, a hybrid method which significantly improves the sparseness of this algorithm, and a method that uses these techniques to make learning of large jigsaws feasible. We provide detailed analysis of how our hybrid inference method leads to significant savings in memory and computation time. To demonstrate the success of our method, we present experimental results applying large jigsaws to an object recognition task. 1.
Statistics of patch offsets for image completion
 In Proc. ECCV
"... Abstract. Image completion involves filling missing parts in images. In this paper we address this problem through the statistics of patch offsets. We observe that if we match similar patches in the image and obtain their offsets (relative positions), the statistics of these offsets are sparsely dis ..."
Abstract

Cited by 7 (1 self)
 Add to MetaCart
(Show Context)
Abstract. Image completion involves filling missing parts in images. In this paper we address this problem through the statistics of patch offsets. We observe that if we match similar patches in the image and obtain their offsets (relative positions), the statistics of these offsets are sparsely distributed. We further observe that a few dominant offsets provide reliable information for completing the image. With these offsets we fill the missing region by combining a stack of shifted images via optimization. A variety of experiments show that our method yields generally better results and is faster than existing stateoftheart methods. 1