This paper provides an algorithm for partitioning grayscale images into disjoint regions of coherent brightness and texture. Natural images contain both textured and untextured regions, so the cues of contour and texture differences are exploited simultaneously. Contours are treated in the intervening contour framework, while texture is analyzed using textons. Each of these cues has a domain of applicability, so to facilitate cue combination we introduce a gating operator based on the texturedness of the neighborhood at a pixel. Having obtained a local measure of how likely two nearby pixels are to belong to the same region, we use the spectral graph theoretic framework of normalized cuts to find partitions of the image into regions of coherent texture and brightness. Experimental results on a wide range of images are shown.

Abstract not found

1 Introduction Traditionally there are two approaches to grouping: region-based methods and contour-based methods. Region-based approaches, such as the Normalized Cut framework [19], have been popular recently. Region-based methods seem to be a natural way to approachthe grouping problem, because (1) regions arise from objects, which are natural entities in grouping; (2) many important cues, such as texture and color, are region-based; (3)region properties are more robust to noise and clutter. Nevertheless, contours, even viewed as boundaries between regions, are themselvesvery important. In many cases boundary contour is the most informative cue in grouping as well as in shape analysis. The intervening contour approach [9] has provided aframework to incorporate contour cues into a region-based framework. However, how to reliably extract contour information, despite years of research, is largely an openproblem. Contour extraction is hard, mainly for the following reasons:

This paper investigates two fundamental problems in computer vision: contour detection and image segmentation. We present state-of-the-art algorithms for both of these tasks. Our contour detector combines multiple local cues into a globalization framework based on spectral clustering. Our segmentation algorithm consists of generic machinery for transforming the output of any contour detector into a hierarchical region tree. In this manner, we reduce the problem of image segmentation to that of contour detection. Extensive experimental evaluation demonstrates that both our contour detection and segmentation methods significantly outperform competing algorithms. The automatically generated hierarchical segmentations can be interactively refined by userspecified annotations. Computation at multiple image resolutions provides a means of coupling our system to recognition applications.

Abstract. Unsupervised learning requires a grouping step that defines which data belong together. A natural way of grouping in images is the segmentation of objects or parts of objects. While pure bottom-up segmentation from static cues is well known to be ambiguous at the object level, the story changes as soon as objects move. In this paper, we present a method that uses long term point trajectories based on dense optical flow. Defining pair-wise distances between these trajectories allows to cluster them, which results in temporally consistent segmentations of moving objects in a video shot. In contrast to multi-body factorization, points and even whole objects may appear or disappear during the shot. We provide a benchmark dataset and an evaluation method for this so far uncovered setting. 1

Abstract not found

Humans beings can easily make abstract judgments of similarity, but current techniques for algorithmically measuring the similarity between two images do so at a very concrete level, measuring simple statistics computed from the rawimage pixels. This dissertation develops and evaluates an evolvable framework for computing image similarity that moves toward more abstract forms of similarity, particularly by allowing the comparison of images based only upon certain significant portions. We begin by formulating and stating the area-matching assumption for concrete visual similarity: Two images are likely to be similar to the extent that they comprise equally matched areas of visually similar materials. We develop an infrastructure to test and explore this approach, and extend it to applications such as classification, image retrieval, and object retrieval. The infrastructure extends from early phases of image processing and analysis, through to multiple-image comparisons and frameworks for applying sophisticated learning algorithms. Throughout we apply the best available tests to evaluate the new techniques and compare them to existing methods. We begin with basic image processing tools that contribute to successful image comparisons. A multi-tiered model-based segmentation algorithm identifies regions of uniform visual

Abstract—This paper investigates two fundamental problems in computer vision: contour detection and image segmentation. We present state-of-the-art algorithms for both of these tasks. Our contour detector combines multiple local cues into a globalization framework based on spectral clustering. Our segmentation algorithm consists of generic machinery for transforming the output of any contour detector into a hierarchical region tree. In this manner, we reduce the problem of image segmentation to that of contour detection. Extensive experimental evaluation demonstrates that both our contour detection and segmentation methods significantly outperform competing algorithms. The automatically generated hierarchical segmentations can be interactively refined by userspecified annotations. Computation at multiple image resolutions provides a means of coupling our system to recognition applications. 1