While different optical flow techniques continue to appear, there has been a lack of quantitative evaluation of existing methods. For a common set of real and synthetic image sequences, we report the results of a number of regularly cited optical flow techniques, including instances of differential, matching, energy-based and phase-based methods. Our comparisons are primarily empirical, and concentrate on the accuracy, reliability and density of the velocity measurements; they show that performance can differ significantly among the techniques we implemented.

Oriented filters are useful in many early vision and image processing tasks. One often needs to apply the same filter, rotated to different angles under adaptive control, or wishes to calculate the filter response at various orientations. We present an efficient architecture to synthesize filters of arbitrary orientations from linear combinations of basis filters, allowing one to adaptively "steer" a filter to any orientation, and to determine analytically the filter output as a function of orientation.

Computing the motions of several moving objects in image sequences involves simultaneous motion analysis and segmentation. This task can become complicated when image motion changes signi#cantly between frames, as with camera vibrations. Such vibrations make tracking in longer sequences harder, as temporal motion constancy can not be assumed. The problem becomes even more di#cult in the case of transparent motions.

Two-dimensional image motion is the projection of the three-dimensional motion of objects, relative to a visual sensor, onto its image plane. Sequences of time-ordered images allow the estimation of projected two-dimensional image motion as either instantaneous image velocities or discrete image displacements. These are usually called the optical flow field or the image velocity field. Provided that optical flow is a reliable approximation to two-dimensional image motion, it may then be used to recover the three-dimensional motion of the visual sensor (to within a scale factor) and the three-dimensional surface structure (shape or relative depth) through assumptions concerning the structure of the optical flow field, the three-dimensional environment and the motion of the sensor. Optical flow may also be used to perform motion detection, object segmentation, time-to-collision and focus of expansion calculations, motion compensated encoding and stereo disparity measurement. We investiga...

Gradient methods are widely used in the computation of optical flow. We discuss extensions of these methods which compute probability distributions of optical flow. The use of distributions allows representation of the uncertainties inherent in the optical flow computation, facilitating the combination with information from other sources. We compute distributed optical flow for a synthetic image sequence and demonstrate that the probabilistic model accounts for the errors in the flow estimates. We also compute distributed optical flow for a real image sequence. 1 Introduction The recovery of motion information from visual input is an important task for both natural and artificial vision systems. Most models for the analysis of visual motion begin by extracting two-dimensional motion information. In particular, computer vision techniques typically compute twodimensional optical flowvectors which describe the motion of each portion of the image in the image plane. Methods for the re...

This thesis describes some new approaches to the representation and analysis of visual motion, as perceived by a biological or machine visual system. We begin by discussing the computation of image motion fields, the projection of motion in the three-dimensional world onto the two-dimensional image plane. This computation is notoriously difficult, and there are a wide variety of approaches that have been developed for use in image processing, machine vision, and biological modeling. We show that a large number of the basic techniques are quite similar in nature, differing primarily in conceptual motivation, and that they each fail to handle a set of situations that occur commonly in natural scenery. The central theme of the thesis is that the failure of these algorithms is due primarily to the use of vector fields as a representation for visual motion. We argue that the translational vector field representation is inherently impoverished and error-prone. Furthermore, there is evidence that a ...

Two paradigms for visual analysis are top-down, starting from high-level models or information about the image, and bottom-up, where little is assumed about the image or objects in it. We explore a local, bottom-up approach to image analysis. We develop operators to identify and classify image junctions, whichcontain important visual cues for identifying occlusion, transparency, and surface bends. Like the human visual system, we begin with the application of linear filters which are oriented in all possible directions. Wedevelop an efficientway to create an oriented filter of arbitrary orientation by describing it as a linear combination of basis filters. This approach to oriented filtering, which we call steerable filters, offers advantages for analysis as well as computation. We design a variety of steerable filters, including steerable quadrature pairs, which measure local energy. We show applications of these filters in orientation and texture analysis, and image representation and enhanc...

A number of researchers have proposed models of early motion sensing based on direction-selective, spatiotemporal linear operators. Others have formalized the problem of measuring optical flow in terms of the spatial and temporal derivatives of stimulus intensity. Recently, the spatiotemporal filter models and the gradient-based methods have been placed into a common framework. In this chapter, we review that framework and we extend it to develop a new model for the computation and representation of velocity information in the visual system. We use the model to simulate psychophysical data on perceived velocity of sine-grating plaid patterns, and to simulate physiological data on responses of simple cells in primary (striate) visual cortex.

A fundamental assumption made in formulating optical-flow algorithms is that motion at any point in an image can be represented as a single pattern component undergoing a simple translation: even complex motion will appear as a uniform displacement when viewed through a sufficiently small window. This assumption fails for a...

More and more low level vision algorithms are being carried out in the spatial frequency domain, using Gabor filters. There are two basic problems concerned with Gabor filterings we will address in this paper. One is the window size problem, in which we will adopt a set of 2D variable window Gabor filters, and compare its performance with those of fixed window filters. We will show that the variable window scheme is more adaptive to image contents, while fixed window schemes may suffer either large errors or instabilities when image contents are changed. The other problem we will address is the stability of amplitude and phase information resulting from convolving the filters with images. We will extend Fleet's 1D phase stability analysis to 2D phase and amplitude stability analysis based upon the assumption of local resemblance of filter outputs to a single sinusoid. Applications on focus quality measurement and 2D correspondence are described, and the results demonstrate improvements...