Contents Thissectionofthecoursenotesisorganizedasfollows: 1.Introductorymaterialforthissection.Thisincludesabriefoverviewofrelatedandcomplimentarymaterialtophotogrammetricmodeling, suchasstructurefrommotion,stereocorrespondence,shapefrom silhouettes,cameracalibration,laserscanning,andimage-basedrendering. 2.Abibliographyofrelatedpapers. 3.Areprintof: PaulE.Debevec,CamilloJ.Taylor,andJitendraMalik.ModelingandRenderingArchitecturefrom Photographs.InSIGGRAPH96,August1996,pp.11-20. 4.NotesonphotogrammetricrecoveryofarchesandsurfacesofrevolutionwrittenbyGeorgeBorshukov. 5.Copiesoftheslidesusedforthepresentation. Moreinformationcanbefoundin[10],[5],and[13],availableat: http://www.cs.berkeley.edu/debevec/Thesis 1 Introduction Thecreationofthree-dimensionalmodelsofexistingarchitecturalsceneswiththeaidofthecomputerhas beencommonplaceforsometime,andtheresultingmodelshavebeenbothentertainingvirtualenvironments aswellasvaluablevisualizationtools.Large-scaleeffortshavepushedthecampusesofI

The problem of efficient, interactive foreground/background segmentation in still images is of great practical importance in image editing. Classical image segmentation tools use either texture (colour) information, e.g. Magic Wand, or edge (contrast) information, e.g. Intelligent Scissors. Recently, an approach based on optimization by graph-cut has been developed which successfully combines both types of information. In this paper we extend the graph-cut approach in three respects. First, we have developed a more powerful, iterative version of the optimisation. Secondly, the power of the iterative algorithm is used to simplify substantially the user interaction needed for a given quality of result. Thirdly, a robust algorithm for “border matting ” has been developed to estimate simultaneously the alpha-matte around an object boundary and the colours of foreground pixels. We show that for moderately difficult examples the proposed method outperforms competitive tools.

In this paper we introduce a new algorithm for image and video texture synthesis. In our approach, patch regions from a sample image or video are transformed and copied to the output and then stitched together along optimal seams to generate a new (and typically larger) output. In contrast to other techniques, the size of the patch is not chosen a-priori, but instead a graph cut technique is used to determine the optimal patch region for any given offset between the input and output texture. Unlike dynamic programming, our graph cut technique for seam optimization is applicable in any dimension. We specifically explore it in 2D and 3D to perform video texture synthesis in addition to regular image synthesis. We present approximative offset search techniques that work well in conjunction with the presented patch size optimization. We show results for synthesizing regular, random, and natural images and videos. We also demonstrate how this method can be used to interactively merge different images to generate new scenes.

We describe an interactive, computer-assisted framework for combining parts of a set of photographs into a single composite picture, a process we call "digital photomontage." Our framework makes use of two techniques primarily: graph-cut optimization, to choose good seams within the constituent images so that they can be combined as seamlessly as possible; and gradient-domain fusion, a process based on Poisson equations, to further reduce any remaining visible artifacts in the composite. Also central to the framework is a suite of interactive tools that allow the user to specify a variety of high-level image objectives, either globally across the image, or locally through a painting-style interface. Image objectives are applied independently at each pixel location and generally involve a function of the pixel values (such as "maximum contrast") drawn from that same location in the set of source images. Typically, a user applies a series of image objectives iteratively in order to create a finished composite. The power of this framework lies in its generality; we show how it can be used for a wide variety of applications, including "selective composites" (for instance, group photos in which everyone looks their best), relighting, extended depth of field, panoramic stitching, clean-plate production, stroboscopic visualization of movement, and time-lapse mosaics.

A new boundary detection approach for shape modeling is presented. It detects the global minimum of an active contour model’s energy between two end points. Initialization is made easier and the curve is not trapped at a local minimum by spurious edges. We modify the “snake” energy by including the internal regularization term in the external potential term. Our method is based on finding a path of minimal length in a Riemannian metric. We then make use of a new efficient numerical method to find this shortest path. It is shown that the proposed energy, though based only on a potential integrated along the curve, imposes a regularization effect like snakes. We explore the relation between the maximum curvature along the resulting contour and the potential generated from the image. The method is capable to close contours, given only one point on the objects’ boundary by using a topology-based saddle search routine. We show examples of our method applied to real aerial and medical images.

This paper describes a new framework for video matting, the process of pulling a high-quality alpha matte and foreground from a video sequence. The framework builds upon techniques in natural image matting, optical flow computation, and background estimation. User interaction is comprised of garbage matte specification if background estimation is needed, and hand-drawn keyframe segmentations into “foreground, ” “background, ” and “unknown”. The segmentations, called trimaps, are interpolated across the video volume using forward and backward optical flow. Competing flow estimates are combined based on information about where flow is likely to be accurate. A Bayesian matting technique uses the flowed trimaps to yield high-quality mattes of moving foreground elements with complex boundaries filmed by a moving camera. A novel technique for smoke matte extraction is also demonstrated. CR Categories: I.3.3 [Computer Graphics]: Picture/Image Generation—Bitmap and framebuffer operations; I.4.6 [Image Processing and Computer Vision]:

We present a new, interactive tool called Intelligent Scissors which we use for image seg-mentation. Fully automated segmentation is an unsolved problem, while manual tracing is inaccu-rate and laboriously unacceptable. However, Intelligent Scissors allow objects within digital images to be extracted quickly and accurately using simple gesture motions with a mouse. When the gestured mouse position comes in proximity to an object edge, a live-wire boundary “snaps” to, and wraps around the object of interest. Live-wire boundary detection formulates boundary detection as an optimal path search in a weighted graph. Optimal graph searching provides mathematically piece-wise optimal bound-aries while greatly reducing sensitivity to local noise or other intervening structures. Robustness is further enhanced with on-the-fly training which causes the boundary to adhere to the specific type of edge currently being followed, rather than simply the strongest edge in the neighborhood. Boundary cooling automatically freezes unchanging segments and automates input of additional seed points. Cooling also allows the user to be much more free with the gesture path, thereby increasing the efficiency and finesse with which boundaries can be extracted. (2) 1.

Live-wire segmentation is a new interactive tool for efficient, accurate, and reproducible boundary extraction which requires minimal user input with a mouse. Optimal boundaries are computed and selected at interactive rates as the user moves the mouse starting from a manually specified seed point. When the mouse position comes in proximity to an object edge, a "live-wire" boundary snaps to, and wraps around the object of interest. Input of a new seed point "freezes" the selected boundary segment, and the process is repeated until the boundary is complete. Two novel enhancements to the basic live wire methodology include boundary cooling and on-the-fly training. Data-driven boundary cooling generates seed points automatically and further reduces user input. On-the-fly training adapts the dynamic boundary to edges of current interest. Using the live wire technique, boundaries are extracted in one-fifth of the time required for manual tracing, but with 4.4 times greater accuracy and 4.8 ...

Many computer vision applications rely on the efficient optimization of challenging, so-called non-submodular, binary pairwise MRFs. A promising graph cut based approach for optimizing such MRFs known as “roof duality” was recently introduced into computer vision. We study two methods which extend this approach. First, we discuss an efficient implementation of the “probing ” technique introduced recently by Boros et al. [5]. It simplifies the MRF while preserving the global optimum. Our code is 400-700 faster on some graphs than the implementation of [5]. Second, we present a new technique which takes an arbitrary input labeling and tries to improve its energy. We give theoretical characterizations of local minima of this procedure. We applied both techniques to many applications, including image segmentation, new view synthesis, superresolution, diagram recognition, parameter learning, texture restoration, and image deconvolution. For several applications we see that we are able to find the global minimum very efficiently, and considerably outperform the original roof duality approach. In comparison to existing techniques, such as graph cut, TRW, BP, ICM, and simulated annealing, we nearly always find a lower energy. 1.

Tracking and modeling people from video sequences has become an increasingly important research topic, with applications including animation, surveillance and sports medicine. In this paper, we propose a model based 3–D approach to recovering both body shape and motion. It takes advantage of a sophisticated animation model to achieve both robustness and realism. Stereo sequences of people in motion serve as input to our system. From these, we extract a 2 1 –D description of the scene and, 2 optionally, silhouette edges. We propose an integrated framework to fit the model and to track the person’s motion. The environment does not have to be engineered. We recover not only the motion but also a full animation model closely resembling the subject. We present results of our system on real sequences and we show the generic model adjusting to the person and following various kinds of motion. Key Words: Shape, 3–D whole-body modeling and tracking, silhouettes 1.