Two images of a single scene/object are related by the epipolar geometry, which can be described by a 3×3 singular matrix called the essential matrix if images' internal parameters are known, or the fundamental matrix otherwise. It captures all geometric information contained in two images, and its determination is very important in many applications such as scene modeling and vehicle navigation. This paper gives an introduction to the epipolar geometry, and provides a complete review of the current techniques for estimating the fundamental matrix and its uncertainty. A well-founded measure is proposed to compare these techniques. Projective reconstruction is also reviewed. The software which we have developed for this review is available on the Internet.

This paper introduces an approach to performance animation that employs video cameras and a small set of retro-reflective markers to create a low-cost, easy-to-use system that might someday be practical for home use. The low-dimensional control signals from the user's performance are supplemented by a database of pre-recorded human motion. At run time, the system automatically learns a series of local models from a set of motion capture examples that are a close match to the marker locations captured by the cameras. These local models are then used to reconstruct the motion of the user as a full-body animation. We demonstrate the power of this approach with real-time control of six different behaviors using two video cameras and a small set of retro-reflective markers. We compare the resulting animation to animation from commercial motion capture equipment with a full set of markers.

We present techniques for improving the speed of robust motion estimation based on random sampling of image features. Starting from Torr and Zisserman's MLESAC algorithm, we address some of the problems posed from both practical and theoretical standpoints and in doing so allow the random search to be replaced by a guided search. Guidance of the search is based on readilyavailable information which is usually discarded, but can significantly reduce the search time. This guided-sampling algorithm is further specialised for tracking of multiple motions, for which results are presented.

Abstract. This paper studies the problem of matching two unsynchronized video sequences of the same dynamic scene, recorded by different stationary uncalibrated video cameras. The matching is done both in time and in space, where the spatial matching can be modeled by a homography (for 2D scenarios) or by a fundamental matrix (for 3D scenarios). Our approach is based on matching space-time trajectories of moving objects, in contrast to matching interest points (e.g., corners), as done in regular feature-based image-to-image matching techniques. The sequences are matched in space and time by enforcing consistent matching of all points along corresponding space-time trajectories. By exploiting the dynamic properties of these space-time trajectories, we obtain sub-frame temporal correspondence (synchronization) between the two video sequences. Furthermore, using trajectories rather than feature-points significantly reduces the combinatorial complexity of the spatial point-matching problem when the search space is large. This benefit allows for matching information across sensors in situations which are extremely difficult when only image-to-image matching is used, including: (a) matching under large scale (zoom) differences, (b) very wide base-line matching, and (c) matching across different sensing modalities (e.g., IR and visible-light cameras). We show examples of recovering homographies and fundamental matrices under such conditions.

This paper studies the problem of sequence-to-sequence alignment, namely establishing correspondences in time and in space between two di erent video sequences of the same dynamic scene. The sequences are recorded by uncalibrated video cameras, which are either stationary or jointly moving, with xed (but unknown) internal parameters and relative inter-camera external parameters. Temporal variations between image frames (such as moving objects or changes in scene illumination) are powerful cues for alignment, which cannot be exploited by standard image-toimage alignment techniques. We show that by folding spatial and temporal cues into a single alignment framework, situations which are inherently ambiguous for traditional image-to-image alignment methods, are often uniquely resolved by sequence-to-sequence alignment. Furthermore, the ability to align and integrate information across multiple video sequences both in time and in space gives rise to new video applications that are not possible when only image-to-image alignment is used. 1

The paradigm of projective vision has recently become popular. In this paper we describe a system for computing camerapositions from an image sequence using projective methods. Projective methods are normally used to deal with uncalibrated images. However, we claim that even when calibration information is available it is often better to use the projective approach. By computing the trilinear tensor it is possible to produce a reliable and accurate set of correspondences. When calibration information is available these correspondences can be sent directly to a photogrammetric program to produce a set of camera positions. We show one way of dealing with the problem of cumulative error in the tensor computation and demonstrate that projective methods can handle surprisingly large baselines, in certain cases one third of the image size. In practice projective methods, along with random sampling algorithms, solve the correspondence problem for many image sequences. To aid in the understanding of this relatively new paradigm we make our binaries available for others on the web. Our software is structured in a way that makes experimentation easy and includes a viewer for displaying the final results.

In this paper, we propose a framework for multi-camera video surveillance. Our framework addresses the detection, represenation, and recognition of motion events. The detection phase handles spatiotemporal data fusion for efficiently and reliably extracting motion trajectories from video. The representation phase summarizes raw trajectory data to construct a hierarchical, invariant, and content-rich representation of the motion events. Finally, the recognition phase deals with learning using imbalanced training datasets and infinitedimensional data that also exhibit temporal ordering. Due to space limit, only the following two components are discussed in the paper: fusing spatio-temporal information from multiple camera sources and characterizing and detecting suspicious surveillance events.

We present a novel multi-planar display system based on an uncalibrated projector-camera pair. Our system exploits the juxtaposition of planar surfaces in a room to create adhoc visualization and display capabilities. In an office setting, for example, a desk pushed against a wall provides two perpendicular surfaces that can simultaneously display elevation and plan views of an architectural model. In contrast to previous room-level projector-camera systems, our method is based on a flexible, minimalist calibration procedure which is tailored to the geometry of the multiplanar surface scenario. Our procedure makes it possible to quickly auto-calibrate the display with a minimum of effort on the part of the user. A number of display configurations can be created on any available planar surfaces using a single commodity projector and camera. The key to our calibration approach is an efficient technique for simultaneously localizing multiple planes and a robust planar metric rectification method which can tolerate a restricted camera field-of-view and requires no special calibration objects. We demonstrate the robustness of our calibration method using real and synthetic images and present several applications of our display system.

We describe a calibration procedure for a multi-camera rig. Consider a large number of synchronized cameras arranged in some space, for example, on the walls of a room looking inwards. It is not necessary for all the cameras to have a common field of view, as long as every camera is connected to every other camera through common fields of view. Switching off the lights and waving a wand with an LED at the end of it, we can capture a very large set of point correspondences (corresponding points are captured at the same time stamp). The correspondences are then used in a large, nonlinear eigenvalue minimization routine whose basis is the epipolar constraint. The eigenvalue matrix encapsulates all points correspondences between every pair of cameras in a way that minimizing the smallest eigenvalue results in the projection matrices, to within a single perspective transformation. In a second step, given additional data from waving a rod with two LEDs (one at each end) the full projection matrices are calculated. The method is extremely accurate—the reprojections of the reconstructed points were within a pixel.

In this paper we show that given two homography matrices for two planes in space, there is a linear algorithm for the rotation and translation between the two cameras, the focal lengths of the two cameras and the plane equations in the space. Using the estimates as an initial guess, we can further optimize the solution by minimizing the difference between observations and reprojections. Experimental results are shown. We also provide a discussion about the relationship between this approach and the Kruppa equation. 1