The α-expansion algorithm [7] has had a significant impact in computer vision due to its generality, effectiveness, and speed. Thus far it can only minimize energies that involve unary, pairwise, and specialized higher-order terms. Our main contribution is to extend α-expansion so that it can simultaneously optimize “label costs ” as well. An energy with label costs can penalize a solution based on the set of labels that appear in it. The simplest special case is to penalize the number of labels in the solution. Our energy is quite general, and we prove optimality bounds for our algorithm. A natural application of label costs is multi-model fitting, and we demonstrate several such applications in vision: homography detection, motion segmentation, and unsupervised image segmentation. Our C++/MATLAB implementation is publicly available. 1. Some Useful Regularization Energies In a labeling problem we are given a set of observations P (pixels, features, data points) and a set of labels L (categories, geometric models, disparities). The goal is to assign each observation p ∈ P a label fp ∈ L such that the joint labeling f minimizes some objective function E(f). Most labeling problems in computer vision are ill-posed and in need of regularization, but the most useful regularizers often make the problem NP-hard. Our work is about how to effectively optimize two such regularizers: a preference for fewer labels in the solution, and a preference for spatial smoothness. Figure 1 suggests how these criteria cooperate to give clean results. Surprisingly, there is no good algorithm to optimize their combination. 1 Our main contribution is a way to simultaneously optimize both of these criteria inside the powerful α-expansion algorithm [7]. Label costs. Start from a basic (unregularized) energy E(f) = ∑ pDp(fp), where optimal fp can each be determined independently from the ‘data costs’. Suppose, however, that we wish to explain the observations using as few unique labels as necessary. We can introduce label costs into E(f) to penalize each unique label that appears in f: E(f) = ∑

This paper proposes a new approach to model-based clustering under prior knowledge. The proposed formulation can be interpreted from two different angles: as penalized logistic regression, where the class labels are only indirectly observed (via the probability density of each class); as finite mixture learning under a grouping prior. To estimate the parameters of the proposed model, we derive a (generalized) EM algorithm with a closed-form E-step, in contrast with other recent approaches to semi-supervised probabilistic clustering which require Gibbs sampling or suboptimal shortcuts. We show that our approach is ideally suited for image segmentation: it avoids the combinatorial nature Markov random field priors, and opens the door to more sophisticated spatial priors (e.g., wavelet-based) in a simple and computationally efficient way. Finally, we extend our formulation to work in unsupervised, semi-supervised, or discriminative modes. 1

We address the problem of segmenting an image sequence into rigidly moving 3D objects. An elegant solution to this problem is the multibody factorization approach in which the measurement matrix is factored into lower rank matrices. Despite progress in factorization algorithms, the performance is still far from satisfactory and in scenes with missing data and noise, most existing algorithms fail. In this paper we propose a method for incorporating 2D non-motion cues (such as spatial coherence) into multibody factorization. We formulate the problem in terms of constrained factor analysis and use the EM algorithm to find the segmentation. We show that adding these cues improves performance in real and synthetic sequences. 1

This paper introduces a formulation which allows using wavelet-based priors for image segmentation. This formulation can be used in supervised, unsupervised, or semisupervised modes, and with any probabilistic observation model (intensity, multispectral, texture). Our main goal is to exploit the well-known ability of wavelet-based priors to model piece-wise smoothness (which underlies state-of-theart methods for denoising, coding, and restoration) and the availability of fast algorithms for wavelet-based processing. The main obstacle to using wavelet-based priors for segmentation is that they’re aimed at representing real values, rather than discrete labels, as needed for segmentation. This difficulty is sidestepped by the introduction of realvalued hidden fields, to which the labels are probabilistically related. These hidden fields, being unconstrained and real-valued, can be given any type of spatial prior, such as one based on wavelets. Under this model, Bayesian MAP segmentation is carried out by a (generalized) EM algorithm. Experiments on synthetic and real data testify for the adequacy of the approach. 1.

Consider spatial data consisting of a set of binary features taking values over a collection of spatial extents (grid cells). We propose a method that simultaneously finds spatial correlation and feature co-occurrence patterns, without any parameters. In particular, we employ the Minimum Description Length (MDL) principle coupled with a natural way of compressing regions. This defines what “good” means: a feature co-occurrence pattern is good, if it helps us better compress the set of locations for these features. Conversely, a spatial correlation is good, if it helps us better compress the set of features in the corresponding region. Our approach is scalable for large datasets (both number of locations and of features). We evaluate our method on both real and synthetic datasets. 1

Abstract not found

The task of assigning labels to pixels is central to computer vision. In automatic segmenta-tion an algorithm assigns a label to each pixel where labels connote a shared property across pixels (e.g. color, bounding contour, texture). Recent approaches to image segmentation have formulated this labeling task as partitioning a graph derived from the image. We use spec-tral segmentation to denote the family of algorithms that seek a partitioning by processing the eigenstructure associated with image graphs. In this thesis we analyze current spectral segmentation algorithms and explain their perfor-mance, both practically and theoretically, on the Normalized Cuts (NCut) criterion. Further, we introduce a novel family of spectral graph partitioning methods, spectral rounding, and ap-ply them to image segmentation tasks. Edge separators of a graph are produced by iteratively reweighting the edges until the graph disconnects into the prescribed number of components. At each iteration a small number of eigenvectors with small eigenvalue are computed and used to determine the reweighting. In this way spectral rounding directly produces discrete solu-tions where as current spectral algorithms must map the continuous eigenvectors to discrete

Abstract. The goal of segmentation is to partition an image into a finite set of regions, homogeneous in some (e.g., statistical) sense, thus being an intrinsically discrete problem. Bayesian approaches to segmentation use priors to impose spatial coherence; the discrete nature of segmentation demands priors defined on discrete-valued fields, thus leading to difficult combinatorial problems. This paper presents a formulation which allows using continuous priors, namely Gaussian fields, for image segmentation. Our approach completely avoids the combinatorial nature of standard Bayesian approaches to segmentation. Moreover, it’s completely general, i.e., itcanbeused in supervised, unsupervised, or semi-supervised modes, with any probabilistic observation model (intensity, multispectral, or texture features). To use continuous priors for image segmentation, we adopt a formulation which is common in Bayesian machine learning: introduction of hidden fields to which the region labels are probabilistically related. Since these hidden fields are real-valued, we can adopt any type of spatial prior for continuous-valued fields, such as Gaussian priors. We show how, under this model, Bayesian MAP segmentation is carried out by a (generalized) EM algorithm. Experiments on synthetic and real data shows that the proposed approach performs very well at a low computational cost. 1

We propose a novel patch-based image representation that is useful because it (1) inherently detects regions with repetitive structure at multiple scales and (2) yields a parameterless hierarchical segmentation. We describe an image by breaking it into coherent regions where each region is well-described (easily reconstructed) by repeatedly instantiating a patch using a set of simple transformations. In other words, a good segment is one that has sufficient repetition of some pattern, and a patch is useful if it contains a pattern that is repeated in the image. Our criterion is naturally expressed by the wellestablished minimum description length (MDL) principle. MDL prefers spatially coherent regions with consistent appearance and avoids parameter tuning. We minimize the description length (in bits) of the image by encoding it with patches. Because a patch is itself an image, we measure its description length by applying the same idea recursively: encode a patch by breaking it into regions described by yet simpler patches. The resulting hierarchy of inter-dependent patches naturally leads to a hierarchical segmentation. We minimize description length over our class of image representations (all patch hierarchies / partitions). We formulate this problem as a recursive multi-label energy. Existing optimization techniques are either inapplicable or get stuck in poor local minima. We propose a new hierarchical fusion (HF) algorithm for energies containing a hierarchy of ‘label costs’. Our algorithm is a contribution in itself and should be useful for this new and difficult class of energies.

We optimize over the set of corrected laplacians (CL) associated with a weighted graph to improve the average case normalized cut (NCut) of a graph. Unlike edge-relaxation SDPs, optimizing over the set CL naturally exploits the matrix sparsity by operating solely on the diagonal. This structure is critical to image segmentation applications because the number of vertices is generally proportional to the number of pixels in the image. CL optimization provides a guiding principle for improving the combinatorial solution over the spectral relaxation, which is important because small improvements in the cut cost often result in significant improvements in the perceptual relevance of the segmentation. We develop an optimization procedure to accommodate prior information in the form of statistical shape models, resulting in a segmentation method that produces foreground regions which are consistent with a parameterized family of shapes. We validate our technique with ground truth on MRI medical images, providing a quantitative comparison against results produced by current spectral relaxation approaches to graph partitioning. 1