Image-based rendering is a powerful new approach for generating real-time photorealistic computer graphics. It can provide convincing animations without an explicit geometric representation. We use the “plenoptic function” of Adelson and Bergen to provide a concise problem statement for image-based rendering paradigms, such as morphing and view interpolation. The plenoptic function is a parameterized function for describing everything that is visible from a given point in space. We present an image-based rendering system based on sampling, reconstructing, and resampling the plenoptic function. In addition, we introduce a novel visible surface algorithm and a geometric invariant for cylindrical projections that is equivalent to the epipolar constraint defined for planar projections.

Leonard McMillan Jr. An Image-Based Approach to Three-Dimensional Computer Graphics (Under the direction of Gary Bishop) The conventional approach to three-dimensional computer graphics produces images from geometric scene descriptions by simulating the interaction of light with matter. My research explores an alternative approach that replaces the geometric scene description with perspective images and replaces the simulation process with data interpolation. I derive an image-warping equation that maps the visible points in a reference image to their correct positions in any desired view. This mapping from reference image to desired image is determined by the center-of-projection and pinhole-camera model of the two images and by a generalized disparity value associated with each point in the reference image. This generalized disparity value, which represents the structure of the scene, can be determined from point correspondences between multiple reference images. The image-warpi...

The visual medium evolved from early paintings to the realistic paintings of the classical era to photographs. The medium of moving imagery started with motion pictures. Television and video recording advanced it to show action “live ” or capture and playback later. In all of the above media, the view of the scene is determined at the transcription time, independent of the viewer. We have been developing a new visual medium called virtualized reality. It delays the selection of the viewing angle till view time, using techniques from computer vision and computer graphics. The visual event is captured using many cameras that cover the action from all sides. The 3D structure of the event, aligned with the pixels of the image, is computed for a few selected directions using a stereo technique. Triangulation and texture mapping enable the placement of a “soft-camera ” to reconstruct the event from any new viewpoint. With a stereo-viewing system, virtualized reality allows a viewer to move freely in the scene, independent of the transcription angles used to record the scene. Virtualized reality has significant advantages over virtual reality. The virtual reality world is typically constructed using simplistic, artificially-created CAD models. Virtualized reality starts with the real world scene and virtualizes it. It is a fully 3D medium as it knows the 3D structure of every point in the image. The applications of virtualized reality are many. Training can become safer and more effective by enabling the trainee to move about freely in a virtualized environment. A whole new entertainment programming can open by allowing the viewer to watch a basketball game while standing on the court or while running with a particular player. In this paper, we describe the hardware and software setup in our “studio ” to make virtualized reality movies. Examples are provided to demonstrate the effectiveness of the system. 1

In traditional stereoscopic displays, the virtual three-dimensional object does not appear to be fixed in space as the viewer's head moves. This apparent motion results from the fact that a correct stereo image can only be formed for a particular viewpoint and interpupillary distance. At other viewpoints, our brain interprets the stereo image as a slightly skewed and rotated version of the original. When moving the head, this skewing of the image is perceived as apparent motion of the object. This apparent

Overview We present an image-based approach for capturing the appearance of a walking or running person so they can be rendered realistically under variable viewpoint and illumination. Considerable work has addressed aspects of postproduction control of viewpoint and illumination of a human performance. Most proposed systems address only one of those two aspects e.g. [Wilburn et al. 2005], [Wenger et al. 2005]. [Theobalt et al. 2005] addressed control both of the viewpoint and illumination, however the approach is challenge by low sampling of both lighting and view dimensions. We take a step toward an image-based approach to obtaining postproduction control over both viewpoint and illumination of cyclic full-body human motion by combining the performance relighting technique of [Wenger et al. 2005] with a novel view generation technique based on a flowed reflectance field. By restricting our consideration to cyclic motion such as walking and running, we are able to acquire a 2D array of views by slowly rotating the subject in front of a 1D vertical array of three high speed cameras and segmenting the data per motion cycle. We then use a combination of light field rendering and view interpolation based on optical flow to render the subject from new viewpoints.

Videoconferencing is going to become attractive for geographically distributed team collaboration, specifically to avoid travelling and to increase flexibility. Against this background this paper presents a next generation system – a 3D videoconference providing immersive tele-presence and natural representation of all participants in a shared virtual meeting space to enhance quality of human-centred communication. This system is based on the principle of a shared virtual table environment, which guarantees correct eye contact and gesture reproduction. The key features of our system are presented and compared to other approaches like tele-cubicles. Furthermore the current system design and details of the real-time hardware and software concept are explained.

Using a combination of techniques from visual representations, view synthesis, and visual-motor model estimation, we present a method for animating movements of an articulated agent (e.g. human or robot arm), without the use of any prior models or explicit 3D information. The information needed to generate simulated images can be acquired either on or off line, by watching the agent doing an arbitrary, possibly unrelated task. We present experimental results synthesizing image sequences of the simulated movement of a human arm and a PUMA 760 robot arm. Control is in either image (camera), motor (joint), or Cartesian world coordinates. We have created a user interface, where a user can input a movement program, and then upon execution, view movies of the (simulated) agent executing the program, along with the instantaneous values of the dynamics variables 1 Introduction View synthesis is the problem of generating images of a scene corresponding to the values of a set of pose parameter...

This paper presents a novel automatic method for view synthesis (or image transfer) from a triplet of uncalibrated images based on trinocular edge matching followed by transfer by interpolation, occlusion detection and correction and finally rendering. The edge-based technique proposed here is of general practical relevance because it overcomes most of the problems encountered in other approaches that either rely upon dense correspondence, work in projective space or need explicit camera calibration. Applications range from immersive media and teleconferencing, image interpolation for fast rendering and compression. 1 Introduction A number of researchers have explored ways of constructing static and temporally varying immersive scenes using real world image data alone. Initial efforts include capturing a large number of viewpoints and use these as an environment map [6] to be applied as a texture on some imaging surface. In this paper we are interested in actually generatin...

In this paper, we propose an approach for construction of a projective 3D voxel space based on epipolar geometry obtained with weak calibration. This concept of voxel space defines the epipolar lines following to the coordinate axis. In the field of computer vision, it is common to reconstruct 3D geometry data based on camera calibration data and disparities of matching points in each image. In the case that the goal of the system is generating the images from another point of view, complete 3D geometry reconstruction is not presently required. However, detecting the consistent matching points in several pairs of images without complete 3D geometry information is difficult. This paper describes the concept of projective 3D voxel, which makes it possible to handle 3D geometric data without complete 3D geometry information. 1

Establishing reliable dense correspondences is crucial for many 3-D and projector-related applications. This paper proposes using temporally encoded patterns to directly obtain the correspondence mapping between a projector and a camera without any searching or calibration. The technique naturally extends to efficiently solve the difficult multiframe correspondence problem across any number of cameras and/or projectors. Furthermore, it automatically determines visibility across all cameras in the system and scales linearly in computation with the number of cameras. Experimental results demonstrate the effectiveness of the proposed technique for a variety of applications. 1.