While different optical flow techniques continue to appear, there has been a lack of quantitative evaluation of existing methods. For a common set of real and synthetic image sequences, we report the results of a number of regularly cited optical flow techniques, including instances of differential, matching, energy-based and phase-based methods. Our comparisons are primarily empirical, and concentrate on the accuracy, reliability and density of the velocity measurements; they show that performance can differ significantly among the techniques we implemented.

Two-dimensional image motion is the projection of the three-dimensional motion of objects, relative to a visual sensor, onto its image plane. Sequences of time-ordered images allow the estimation of projected two-dimensional image motion as either instantaneous image velocities or discrete image displacements. These are usually called the optical flow field or the image velocity field. Provided that optical flow is a reliable approximation to two-dimensional image motion, it may then be used to recover the three-dimensional motion of the visual sensor (to within a scale factor) and the three-dimensional surface structure (shape or relative depth) through assumptions concerning the structure of the optical flow field, the three-dimensional environment and the motion of the sensor. Optical flow may also be used to perform motion detection, object segmentation, time-to-collision and focus of expansion calculations, motion compensated encoding and stereo disparity measurement. We investiga...

The computahon of optical flow rehes on merg. ,ng znformat,on avadable over an zmage patch to form an estimate of D mage veloct!t at a point. Ths merging process rases a host of ssues, which include the treatment of outhers m component ve- !oc*t!t measurements and the modehng of mulhple motions wathm a patch whzch arse from occlusion boundaries or transparency. We present a new ap- proach for deahno wth these ssues. which s based Proc. CVPR'93, New York, June 1993 2 a c Figure 2: Multiple motion constraint lines for the region in Figure I (see text).

The measurement of image disparity is a fundamental precursor to binocular depth estimation. Recently, Jenkin and Jepson (1988) and Sanger (1988) described promising methods based on the output phase behaviour of band-pass Gabor filters. Here we discuss further justification for such techniques based on the stability of band-pass phase behaviour as a function of typical distortions that exist between left and right views. In addition, despite this general stability, we show that phase signals are occasionally very sensitive to spatial position and variations in scale, in which cases incorrect measurements occur. We find that the primary cause for this instability is the existence of singularities in phase signals. With the aid of the local frequency of the filter output (provided by the phase derivative) and the local amplitude information, the regions of phase instability near the singularities are detected so that potentially incorrect measurements can be identified. In addition, we ...

We have developed a real-time, view-based gesture recognition system. Optical flow is estimated and segmented into motion blobs. Gestures are recognized using a rule-based technique based on characteristics of the motion blobs such as relative motion and size. Parameters of the gesture (e.g., frequency) are then estimated using context specific techniques. The system has been applied to create an interactive environment for children. 1

ii About the cover The figure on the cover shows a visualization of a symmetric tensor in three dimensions, G = λ1ê1ê T 1 + λ2ê2ê T 2 + λ3ê3ê T 3 The object in the figure is the sum of a spear, a plate and a sphere. The spear describes the principal direction of the tensor λ1ê1ê T 1, where the length is proportional to the largest eigenvalue, λ1. The plate describes the plane spanned by the eigenvectors corresponding to the two largest eigenvalues, λ2(ê1ê T 1 + ê2ê T 2). The sphere, with a radius proportional to the smallest eigenvalue, shows how isotropic the tensor is, λ3(ê1ê T 1 + ê2ê T 2 + ê3ê T 3). The visualization is done using AVS [WWW94]. I am very grateful to Johan Wiklund for implementing the tensor viewer module used. This thesis deals with filtering of multidimensional signals. A large part of the thesis is devoted to a novel filtering method termed “Normalized convolution”. The method performs local expansion of a signal in a chosen filter basis which

We propose a Bayesian framework for representing and recognizing local image motion in terms of two basic models: translational motion and motion boundaries. Motion boundaries are represented using a non-linear generative model that explicitly encodes the orientation of the boundary, the velocities on either side, the motion of the occluding edge over time, and the appearance/disappearance of pixels at the boundary. We represent the posterior probability distribution over the model parameters given the image data using discrete samples. This distribution is propagated over time using a particle filtering algorithm. To efficiently represent such a high-dimensional space we initialize samples using the responses of a low-level motion discontinuity detector. The formulation and computational model provide a general probabilistic framework for motion estimation with multiple, non-linear, models.

The image motion field for an observer moving through a static environment depends on the observer's translational and rotational velocities along with the distances to surface points. Given such a motion field as input we have recently introduced subspace methods for the recovery of the observer's motion and the depth structure of the scene. This class of methods involve splitting the equations describing the motion field into separate equations for the observer's translational direction, the rotational velocity, and the relative depths. The resulting equations can then be solved successively, beginning with the equations for the translational direction. Here we concentrate on this first step. In earlier work, a linear method was shown to provide a biased estimate of the translational direction. We discuss the source of this bias and show how it can be effectively removed. The consequence is that the observer's velocity and the relative depths to points in the scene can all be recover...

: Working toward ecient (real-time) implementations of optical ow methods, we have applied simple recursive lters to achieve temporal smoothing and dierentiation of image intensity, and to compute 2d ow from component velocity constraints using spatiotemporal least-squares minimization. Accuracy in simulation is similar to that obtained in the study by Barron et al. [3], while requiring much less storage, less computation, and shorter delays. 1 Introduction Many methods exist for computing optic ow, but few currently run at frame rates on reasonably priced, conventional hardware. The goal of this paper is to outline simplications to a successful gradient-based approach that reduce computational expense with little degradation in accuracy. Our specic concerns include temporal smoothing and dierentiation of image intensity, and temporal integration of component velocity constraints to solve for 2d velocity. More generally, we are working toward ecient implementations of dierent...

Non-Fourier motion is now commonplace in research on visual motion perception, yet lacks a computational framework. This paper examines this issue based on the observation that many non-Fourier motion stimuli have a simple characterization in the frequency domain, in terms of oriented power distributions that lie along lines (or planes) that do not pass through the origin. This provides a unifying theoretical framework for a very diverse class of non-Fourier phenomena. It also allows us to examine some central issues concerning the computational nature of non-Fourier models, and naturally occurring sources of non-Fourier motion. For example, it is shown that the orientation of power in frequency domain corresponds to the velocity of a multiplicative envelope, and may arise as a restricted form of lighting effects, translucency or occlusion. We also show that both the location and orientation of spectral power may be extracted from the phase and amplitude output of band-pass filters, co...