No feature-based vision system can work unless good features can be identified and tracked from frame to frame. Although tracking itself is by and large a solved problem, selecting features that can be tracked well and correspond to physical points in the world is still hard. We propose a feature selection criterion that is optimal by construction because it is based on how the tracker works, and a feature monitoring method that can detect occlusions, disocclusions, and features that do not correspond to points in the world. These methods are based on a new tracking algorithm that extends previous Newton-Raphson style search methods to work under affine image transformations. We test performance with several simulations and experiments.

Ordinary cameras gather light across the area of their lens aperture, and the light striking a given subregion of the aperture is structured somewhat differently than the light striking an adjacent subregion. By analyzing this optical structure, one can infer the depths of objects in the scene, i.e., one can achieve "single lens stereo." We describe a novel camera for performing this analysis. It incorporates a single main lens along with a lenticular array placed at the sensor plane. The resulting "plenoptic camera" provides information about how the scene would look when viewed from a continuum of possible viewpoints bounded by the main lens aperture. Deriving depth information is simpler than in a binocular stereo system because the correspondence problem is minimized. The camera extracts information about both horizontal and vertical parallax, which improves the reliability of the depth estimates.

This thesis describes some new approaches to the representation and analysis of visual motion, as perceived by a biological or machine visual system. We begin by discussing the computation of image motion fields, the projection of motion in the three-dimensional world onto the two-dimensional image plane. This computation is notoriously difficult, and there are a wide variety of approaches that have been developed for use in image processing, machine vision, and biological modeling. We show that a large number of the basic techniques are quite similar in nature, differing primarily in conceptual motivation, and that they each fail to handle a set of situations that occur commonly in natural scenery. The central theme of the thesis is that the failure of these algorithms is due primarily to the use of vector fields as a representation for visual motion. We argue that the translational vector field representation is inherently impoverished and error-prone. Furthermore, there is evidence that a ...

This papers presents a framework for detecting and tracking moving objects in a sequence of images. Using a statistical approach, where the inter-frame di#erence is modeled by a mixture of two Laplacian or Gaussian distributions, and an energy minimization based approach, we reformulate the motion detection and tracking problem as a front propagation problem. The Euler-Lagrange equation of the designed energy functional is #rst derived and the #ow minimizing the energy is then obtained. Following the work by Caselles et al [11] and Malladi et al [23, 24] the contours to be detected and tracked are modeled as geodesic active contours evolving toward the minimum of the designed energy, under the in#uence of internal and external image dependent forces. Using the level set formulation scheme of Osher and Sethian [29], complex curves can be detected and tracked and topological changes for the evolving curves are naturally managed. To reduce the computational cost required by a direct implementation of the formulation scheme of Osher and Sethian [29], a new approach exploiting aspects from the classical Narrow Band [3] and Fast Marching [33] methods is proposed and favorably compared to them. In order to further reduce the CPU time, a multi-scale approach has also been considered. Very promising experimental results are provided using real video sequences.

In this paper we address two important problems in motion analysis: the detection of moving objects and their localization. A statistical approach is adopted in order to formulate these problems. For the first, the inter-frame difference is modelized by a mixture of two zero-mean generalized Gaussian distributions, and a Gibbs random field is used for describing the label set. A new method to determine the regularization parameter is proposed, based on a voting technique. This method is also modelized using a statistical framework. The solution of the second problem is based on the observation of only two successive frames. Using the results of change detection an adaptive statistical model for the couple of image intensities is identified. For each problem two different multiscale algorithms are evaluated, and the labeling problem is solved using either ICM (Iterated Conditional Modes) or HCF (Highest Confidence First) algorithms. For illustrating the efficiency of the proposed approach, experimental results are provided using synthetic and real video sequences.

Two important problems in motion analysis are addressed in this paper: change detection and moving object location. For the first problem, the inter-frame difference is modelized by a mixture of Laplacian distributions, a Gibbs random field is used for describing the label field, and HCF (Highest Confidence First) algorithm is used for solving the resulting optimization problem. The solution of the second problem is based on the observation of two successive frames alone. Using the results of change detection an adaptive statistical model for the couple of image intensities is identified. Then the labeling problem is solved using HCF algorithm. Results on real image sequences illustrate the efficiency of the proposed method. 1. Introduction Detection and location of moving objects in an image sequence is a very important task in numerous applications of Computer Vision, including object tracking, fixation and 2-D/3-D motion estimation. For a stationary observer, detection is often ba...

Hierarchical decomposition of images and their relationship with motion fields continues to be a hotly pursued topic, and the role of backward motion information in coding is beginning to capture the interest of video coding community. This paper simultaneously addresses some of the fundamental issues in multi-resolution and backward motion systems. From a coding viewpoint, a multi-resolution motion hierarchy should be coupled with an estimation system that deals with a maximally subsampled wavelet decomposition of the frames, to avoid redundancy of representation. Through a frequency domain argument, we expose the difficulties associated with such an approach, and in fact show that a band-to-band motion compensated estimation in a wavelet domain is not possible. This analysis leads to an alternative approach for estimation of detail bands. The resulting estimation errors were coded through a zerotree quantizer. We circumvented the causality problem associated with determination of zer...

In this paper we address two problems crucial to motion analysis: the detection of moving objects and their localisation. Statistical and level set approaches are adopted in formulating these problems. For the change detection problem, the inter-frame difference is modelled by a mixture of two zero-mean Laplacian distributions. At first, statistical tests using criteria with negligible error probability are used for labelling as changed or unchanged as many sites as possible. All the connected components of the labelled sites are used thereafter as region seeds, which give the initial level sets for which velocity fields for label propagation are provided. We introduce a new multi-label fast marching algorithm for expanding competitive regions. The solution of the localisation problem is based on the map of changed pixels previously extracted. The boundary of the moving objects...

We describe a new form of representation of image velocities, which does not rely on vector fields. For each local spatio-temporal region of the input image, we desire a function over the space of velocities describing the presence of a given velocity in that region. This function maybeinterpreted as a probability distribution over velocity, although it is not necessary to do so. A primary advantage of this representation is that it is capable of representing more than one velocityat a given image location. A multi-modal distribution indicates the presence of multiple motions. Such situations occur frequently in natural scenes near occlusion boundaries, and in situations of transparency. We develop an example of this type of representation through a series of modifications of current differential approaches to motion estimation. We define an angular version of the standard gradient constraint equation, and then extend this to representmultiple motions. The derivation is first d...