In this paper we consider the problem of computing the 3D shape of an unknown, arbitrarily-shaped scene from multiple photographs taken at known but arbitrarilydistributed viewpoints. By studying the equivalence class of all 3D shapes that reproduce the input photographs, we prove the existence of a special member of this class, the photo hull, that (1) can be computed directly from photographs of the scene, and (2) subsumes all other members of this class. We then give a provably-correct algorithm, called Space Carving, for computing this shape and present experimental results on complex real-world scenes. The approach is designed to (1) build photorealistic shapes that accurately model scene appearance from a wide range of viewpoints, and (2) account for the complex interactions between occlusion, parallax, shading, and their effects on arbitrary views of a 3D scene. 1.

Creating geometrically correct and complete 3D models of complex environments remains a difficult problem. Techniques for 3D digitizing and modeling have been rapidly advancing over the past few years although most focus on single objects or specific applications such as architecture and city mapping. The ability to capture details and the degree of automation vary widely from one approach to another. One can safely say that there is no single approach that works for all types of environment and at the same time is fully automated and satisfies the requirements of every application. In this paper we show that for complex environments, those composed of several objects with various characteristics, it is essential to combine data from different sensors and information from different sources. Our approach combines models created from multiple images, single images, and range sensors. It can also use known shapes, CAD, existing maps, survey data, and GPS. 3D points in the image-based models are generated by photogrammetric bundle adjustment with or without self-calibration depending on the image and point configuration. Both automatic and interactive procedures are used depending on the availability of reliable automated process. Producing high quality and accurate models, rather than full automation, is the goal. Case studies in diverse environments are used to demonstrate that all the aforementioned features are needed for environments with a significant amount of complexity.

In this paper we consider the problem of computing the 3D shape of an unknown, arbitrarily-shaped scene from multiple color photographs taken at known but arbitrarilydistributed viewpoints. By studying the equivalence class of all 3D shapes that reproduce the input photographs, we prove the existence of a special member of this class, the maximal photo-consistent shape, that (1) can be computed from an arbitrary volume that contains the scene, and (2) subsumes all other members of this class. We then give a provably-correct algorithm for computing this shape and present experimental results from applying it to the reconstruction of a real 3D scene from several photographs. The approach is specifically designed to (1) build 3D shapes that allow faithful reproduction of all input photographs, (2) resolve the complex interactions between occlusion, parallax, shading, and their effects on arbitrary collections of photographs of a scene, and (3) follow a "least commitment" approach to 3D sh...

: We introduce the notion of oriented projective reconstruction (OPR). We show that, in contrary to common belief, it is possible to obtain more than a projective reconstruction (PR) of a scene from uncalibrated real cameras, namely OPR. This is enabled by knowing that a real camera sees only points in front of it. The defining property of OPR is that the plane that is in an underlying Euclidean reconstruction at infinity does not intersect the convex hull of the reconstructed points in OPR. This is generally not true for PR. Thus, OPR can be viewed as a step between affine reconstruction (when the plane at infinity projects to infinity) and PR (when the position of the plane at infinity is unconstrained). The important practical consequence is that OPR preserves the convex hull, and the reconstructed scene is "topologically correct" and it can be e.g. rendered with hidden surfaces removed correctly. 1 Introduction Let us consider reconstructing a scene from image points obtained by...

the CMP. Their contribution is clearly apparent from the authorship of the cited publications. The original idea of view interpolation for my thesis is due to Roger David Hersch. He transferred it to V'aclav, and organized the partial financial support of my work during the first years. I thank V'aclav Hlav'ac, Radim S'ara, and George Matas from the CMP for the careful reading of the manuscript. I acknowledge the following financial support of the research of mine and the CMP team: the Grant Agency of the Czech Republic, grants 102/93/0954, 102/95/1378, 102/97/0480, 102/97/0855, and 201/97/0437; the Swiss National Fund, grant 83H-036863; the Laboratory of Peripheral Devices EPFL Lausanne, the Interview project; the Czech Ministry of Education, grants VS96049 and OK212/97; the Czech Technical University, grant 38196; the European Union, grant Copernicus CP941068. IV Mathematical Notation I n n \Theta n identity matrix<

We introduce a novel approach to capturing light field with lensless cameras. By moving the cameras back and forth, we capture a set of images. We show that it is possible to reconstruct the light field from these blurry images. The problem is formulized in a way similar to computer tomography, so that the light field can be reconstructed using existing algorithms. The light field can then be used to render 3D scenes. Synthetic examples are presented to show the effectiveness of the proposed method. 1.

In this paper, we apply generalized sampling to imagebased rendering (IBR) data, more specifically, the lightfield. We show that in theory the lowest sampling rate of lightfield when we use generalized sampling can be as low as half of that when we use rectangular sampling. However, in practice rectangular sampling has several advantages over generalized sampling. We analyze the pros and cons for each sampling approach, and explain why in practice rectangular sampling is still more preferable.

Image-based rendering (IBR) has attracted a lot of research interest recently. In this paper, we survey the various techniques developed for IBR, including representation, sampling and compression. The goal is to provide an overview of research for IBR in a complete and systematic manner. We observe that essentially all the IBR representations are derived from the plenoptic function, which is seven dimensional and difficult to handle. We classify various IBR representations into two categories based on how the plenoptic function is simplified, namely restraining the viewing space and introducing source descriptions. In the former category, we summarize six common assumptions that were often made in various approaches and discuss how the dimension of the plenoptic function can be reduced based on these assumptions. In the latter category, we further categorize the methods based on what kind of source description was introduced, such as scene geometry, texture map or reflection model. Sampling and compression are also discussed respectively for both categories.

Abstract — Active multi-camera networks have a large and growing application base. On one end of the spectrum, active camera networks are being used to enhance the rendering or modeling of a single scene. Here, the many simultaneous views can be used to render a synthetic real time view even in a dynamically changing environment. The technical challenges include depth estimation within the scene, image correlation between multiple camera views, and manipulating the camera nodes in order to improve the rendered image. Active multicamera networks are also being used to enhance surveillance applications for which a large area needs to be monitored. In these systems, a primary focus is tracking objects both within a single video feed as well as throughout a collection of video feeds. Active components are used to monitor larger areas and provide more continuous coverage of moving targets. Regardless of the application, real time processing constraints and bandwidth limitations constitute a significant problem with large networked camera systems. This paper presents an overview of these two highly researched application areas in the context of active multicamera networks. For each, a breakdown of the typical approaches is presented along with a survey of real systems that implement them. We conclude with a brief discussion of the major research areas and future application potential combining the two technologies. Index Terms—image-based rendering, multi-camera, Surveillance, active sensor networks