In this paper we describe a new technique for general purpose interactive segmentation of N-dimensional images. The user marks certain pixels as “object” or “background” to provide hard constraints for segmentation. Additional soft constraints incorporate both boundary and region information. Graph cuts are used to find the globally optimal segmentation of the N-dimensional image. The obtained solution gives the best balance of boundary and region properties among all segmentations satisfying the constraints. The topology of our segmentation is unrestricted and both “object” and “background” segments may consist of sev-eral isolatedparts. Some experimental results are presented in the context ofphotohideo editing and medical image seg-mentation. We also demonstrate an interesting Gestalt example. A fast implementation of our segmentation method is possible via a new mar-$ow algorithm in [2].

Combinatorial graph cut algorithms have been successfully applied to a wide range of problems in vision and graphics. This paper focusses on possibly the simplest application of graph-cuts: segmentation of objects in image data. Despite its simplicity, this application epitomizes the best features of combinatorial graph cuts methods in vision: global optima, practical efficiency, numerical robustness, ability to fuse a wide range of visual cues and constraints, unrestricted topological properties of segments, and applicability to N-D problems. Graph cuts based approaches to object extraction have also been shown to have interesting connections with earlier segmentation methods such as snakes, geodesic active contours, and level-sets. The segmentation energies optimized by graph cuts combine boundary regularization with region-based properties in the same fashion as Mumford-Shah style functionals. We present motivation and detailed technical description of the basic combinatorial optimization framework for image segmentation via s/t graph cuts. After the general concept of using binary graph cut algorithms for object segmentation was first proposed and tested in Boykov and Jolly (2001), this idea was widely studied in computer vision and graphics communities. We provide links to a large number of known extensions based on iterative parameter re-estimation and learning, multi-scale or hierarchical approaches, narrow bands, and other techniques for demanding photo, video, and medical applications.

Abstract. In this paper, we report an active contour algorithm that is capable of using prior shapes. The energy functional of the contour is modified so that the energy depends on the image gradient as well as the prior shape. The model provides the segmentation and the transformation that maps the segmented contour to the prior shape. The active contour is able to find boundaries that are similar in shape to the prior, even when the entire boundary is not visible in the image (i.e., when the boundary has gaps). A level set formulation of the active contour is presented. The existence of the solution to the energy minimization is also established. We also report experimental results of the use of this contour on 2d synthetic images, ultrasound images and fMRI images. Classical active contours cannot be used in many of these images.

Abstract. An N-dimensional image is divided into “object ” and “background” segments using a graph cut approach. A graph is formed by connecting all pairs of neighboring image pixels (voxels) by weighted edges. Certain pixels (voxels) have to be a priori identified as object or background seeds providing necessary clues about the image content. Our objective is to find the cheapest way to cut the edges in the graph so that the object seeds are completely separated from the background seeds. If the edge cost is a decreasing function of the local intensity gradient then the minimum cost cut should produce an object/background segmentation with compact boundaries along the high intensity gradient values in the image. An efficient, globally optimal solution is possible via standard min-cut/max-flow algorithms for graphs with two terminals. We applied this technique to interactively segment organs in various 2D and 3D medical images. 1

Abstract. All previous geometric active contour models that have been formulated as gradient flows of various energies use the same L 2-type inner product to define the notion of gradient. Recent work has shown that this inner product induces a pathological Riemannian metric on the space of smooth curves. However, there are also undesirable features associated with the gradient flows that this inner product induces. In this paper, we reformulate the generic geometric active contour model by redefining the notion of gradient in accordance with Sobolev-type inner products. We call the resulting flows Sobolev active contours. Sobolev metrics induce favorable regularity properties in their gradient flows. In addition, Sobolev active contours favor global translations, but are not restricted to such motions; they are also less susceptible to certain types of local minima in contrast to traditional active contours. These properties are particularly useful in tracking applications. We demonstrate the general methodology by reformulating some standard edge-based and regionbased active contour models as Sobolev active contours and show the substantial improvements gained in segmentation.

... alternative to pure automatic segmentation in many applications. While automatic segmentation can be very challenging, a small amount of user input can often resolve ambiguous decisions on the part of the algorithm. In this work, we devise a graph cut algorithm for interactive segmentation which incorporates shape priors. While traditional graph cut approaches to interactive segmentation are often quite successful, they may fail in cases where there are diffuse edges, or multiple similar objects in close proximity to one another. Incorporation of shape priors within this framework mitigates these problems. Positive results on both medical and natural images are demonstrated.

This paper presents an e#cient technique for unsupervised content-based segmentation in stereoscopic video sequences by appropriately combined di#erent content descriptors in a hierarchical framework. Three main modules are involved in the proposed scheme; extraction of reliable depth information, image partition into color and depth regions and a constrained fusion algorithm of color segments using information derived from the depth map. In the #rst module, each stereo pair is analyzed and the disparity #eld and depth map are estimated. Occlusion detection and compensation are also applied for improving the depth map estimation. In the following phase, color and depth regions are created using a novel complexity-reducingmultiresolution implementation of the Recursive Shortest Spanning Tree algorithm #M-RSST#. While depth segments provide a coarse representation of the image content, color regions describe very accurately object boundaries. For this reason, in the #nal phase, a new segmentation fusion algorithm is employed which projects color segments onto depth segments. Experimental results are presented which exhibit the e#ciency of the proposed scheme as content-based descriptor, even in case of images with complicated visual content

Shape models used for the segmentation of 3D image data often suffer from high instability of shape. Current approaches to avoid this instability often result in models with high computation times and few possibilities for interaction and modelling.

Perceptual organization refers to the process of organizing sensory input into coherent and interpretable perceptual structures. This process is challenging due to the chicken-and-egg nature between the various sub-processes such as image segmentation, figure-ground segregation and object recognition. Low-level processing requires the guidance of high-level knowledge to overcome noise; while high-level processing relies on low-level processes to reduce the computational complexity. Neither process can be sufficient on its own. Consequently, any system that carries out these processes in a sequence is bound to be brittle. An alternative system is one in which all processes interact with each other simultaneously. In this thesis, we develop a set of simple yet realistic interactive processing models for perceptual organization. We model the processing in the framework of spectral graph theory, with a criterion encoding the overall goodness of perceptual organization. We derive fast solutions for near-global optima of the criterion, and demonstrate the efficacy of the models on segmenting a wide range of real images. Through these models, we are able to capture a variety of perceptual phenomena: a unified treatment of various grouping, figure-ground and depth cues to produce popout, region segmentation and depth segregation in one step; and a unified framework for integrating bottom-up and top-down information to produce an object segmentation from spatial and object attention. We achieve these goals by empowering current spectral graph methods with a principled solution for multiclass spectral graph partitioning; expanded repertoire of grouping cues to include similarity, dissimilarity and ordering relationships; a theory for integrating sparse grouping cues; and a model ...

The video coding standard MPEG-4 is enabling content-based functionalities by the introduction of video object planes (VOP's) which represent semantically meaningful objects. In this paper, a novel fast, unsupervised semantic segmentation scheme is presented for stereoscopic sequences, which utilizes the provided depth information. Each stereo pair is first analyzed and the disparity field and occluded areas are estimated. Then a multiresolution implementation of the RSST segmentation algorithm is applied to the depth map for extracting the depth segments. For each depth segment, except the last, feature points are generated on its contour and a motion geometric space (MGS) for every initial point is defined. Afterwards one point per MGS is selected, which satisfies predefined intensity and curvature constraints so that the object boundaries are accurately extracted. Experimental results are presented to indicate the good performance of the proposed scheme on real life stereoscopic video sequences.