The availability of quantitative online benchmarks for low-level vision tasks such as stereo and optical flow has led to significant progress in the respective fields. This paper introduces such a benchmark for image matting. There are three key factors for a successful benchmarking system: (a) a challenging, high-quality ground truth test set; (b) an online evaluation repository that is dynamically updated with new results; (c) perceptually motivated error functions. Our new benchmark strives to meet all three criteria. We evaluated several matting methods with our benchmark and show that their performance varies depending on the error function. Also, our challenging test set reveals problems of existing algorithms, not reflected in previously reported results. We hope that our effort will lead to considerable progress in the field of image matting, and welcome the reader to visit our benchmark at www.alphamatting.com. 1.

Abstract. Many computer vision problems such as object segmentation or reconstruction can be formulated in terms of labeling a set of pixels or voxels. In certain scenarios, we may know the number of pixels or voxels which can be assigned to a particular label. For instance, in the reconstruction problem, we may know size of the object to be reconstructed. Such label count constraints are extremely powerful and have recently been shown to result in good solutions for many vision problems. Traditional energy minimization algorithms used in vision cannot handle label count constraints. This paper proposes a novel algorithm for minimizing energy functions under constraints on the number of variables which can be assigned to a particular label. Our algorithm is deterministic in nature and outputs ε-approximate solutions for all possible counts of labels. We also develop a variant of the above algorithm which is much faster, produces solutions under almost all label count constraints, and can be applied to all submodular quadratic pseudoboolean functions. We evaluate the algorithm on the two-label (foreground/background) image segmentation problem and compare its performance with the stateof-the-art parametric maximum flow and max-sum diffusion based algorithms. Experimental results show that our method is practical and is able to generate impressive segmentation results in reasonable time. 1

This paper addresses the problem of extracting an alpha matte from a single photograph given a user-defined trimap. A crucial part of this task is the color modeling step where for each pixel the optimal alpha value, together with its confidence, is estimated individually. This forms the data term of the objective function. It comprises of three steps: (i) Collecting a candidate set of potential fore- and background colors; (ii) Selecting high confidence samples from the candidate set; (iii) Estimating a sparsity prior to remove blurry artifacts. We introduce novel ideas for each of these steps and show that our approach considerably improves over state-of-the-art techniques by evaluating it on a large database of 54 images with known high-quality ground truth. 1

In this paper we considerably improve on a state-of-theart alpha matting approach by incorporating a new prior which is based on the image formation process. In particular, we model the prior probability of an alpha matte as the convolution of a high-resolution binary segmentation with the spatially varying point spread function (PSF) of the camera. Our main contribution is a new and efficient deconvolution approach that recovers the prior model, given an approximate alpha matte. By assuming that the PSF is a kernel with a single peak, we are able to recover the binary segmentation with an MRF-based approach, which exploits flux and a new way of enforcing connectivity. The spatially varying PSF is obtained via a partitioning of the image into regions of similar defocus. Incorporating our new prior model into a state-of-the-art matting technique produces results that outperform all competitors, which we confirm using a publicly available benchmark. 1.

Recently, the quantitative evaluation of interactive single image matting techniques has become possible by the introduction of high-quality ground truth datasets. However, quantitative comparisons conducted in previous work are based on error metrics (e.g. sum of absolute differences) that are not necessarily correlated to the visual quality of the image as perceived by the user. This motivates research to better understand the perception of errors inherent to matting algorithms, in order to provide the ground for a future design of error metrics that better reflect the subjective impression of the human observer. In this work we gain novel insights into the perception of errors due to imperfect matting results. To investigate these errors, we compare two recent state-of-the-art matting algorithms in a user study. We use an eye-tracker to reveal details of the decision making of the users. The data acquired in the user study show a considerable correlation between expert knowledge in photography and the ability of the user to detect errors in the image. This is also reflected in the eye-tracking data which reveals different types of scanning paths dependent on the experience of the user. Keywords: Eye-tracking, image matting, user study 1.

This paper proposes a new multi-scale energy minimization algorithm which can be used to efficiently solve large scale labelling problems in computer vision. The basic modus operandi of any multi-scale method involves the construction of a smaller problem which can be solved efficiently. The solution of this problem is used to obtain a partial labelling of the original energy function, which in turn allows us to minimize it by solving its (much smaller) projection. We propose the use of new techniques for both the construction of the smaller problem, and the extraction of a partial solution. We demonstrate our method on the problem of interactive image segmentation. Traditional multi-scale approaches for segmentation extract a partial solution using an image band around the boundaries of the object segmentation obtained by minimizing the smaller problem. This strategy fails on objects with fine structures and complex topologies. In contrast, our novel approach uses a min-marginal based uncertainty measure which allows us to handle such objects. Experiments show that our techniques result in solutions with low pixel labelling error. Furthermore, they take the same or less amount of computation compared to traditional multi-scale techniques. 1.

In this paper, we propose an automatic method to extract a foreground object captured from multiple viewpoints. We consider the foreground object is within the visual hull of camera field of views. By exploring the multi-view geometric relationship and color measurements of the input images, we can estimate the foreground segmentations as well as their fractional boundaries. To facilitate efficient computation and high quality mattes, we adopt a two-phase approach. The first phase of our algorithm provides quick and rough binary segmentations of the foreground object using graph-cut; the second phase refines the segmentation boundaries using matting. Our result is the high quality alpha mattes of the foreground object consistently across all different viewpoints. We demonstrate the effectiveness of our method using challenging examples. Index Terms — co-segmentation, multiple views, matting 1.

Image matting aims to extract a foreground object from a single natural image by recovering the partial transparency and corresponding color of the foreground object at each pixel in the image. The resulting transparency map is thereby denoted as alpha matte. The matting problem is severely ill-posed, and in this thesis we focus on matting approaches that utilize user interaction to make the problem tractable. There are three fundamental challenges in interactive image matting research that are addressed in this thesis: (i) Providing a fast and intuitive user interface; (ii) finding a good cost function for matting; and (iii) providing a benchmark that allows a quantitative comparison of matting results. In most previous approaches the user interacts with the algorithm by drawing an accurate trimap, which is a partition of the image into foreground, background and unknown regions. An accurate trimap is very tedious to create manually, hence we follow recent work and aim to automatically generate a trimap from very little user input. The novelty of our approach lies in a new cost function that describes the goodness of a trimap solution. Our cost function considers several image cues and incorporates four different types of priors that are used to regularize the result. We show that our method is fast and produces accurate results.