This paper presents a new approach to the estimation of two-dimensional motion vector fields from time-varying images. The approach is stochastic, both in its formulation and in the solution method. The formulation involves the specification of a deterministic structural model, along with stochastic observation and motion field models. Two motion models are proposed: a globally smooth model based on vector Markov random fields and a piecewise smooth model derived from coupled vector-binary Markov random fields. Two estimation criteria are studied. In the Maximum A Posteriori Probability (MAP) estimation the a posteriori probability of motion given data is maximized, while in the Minimum Expected Cost (MEC) estimation the expectation of a certain cost function is minimized. The MAP estimation is performed via simulated annealing, while the MEC algorithm performs iteration-wise averaging. Both algorithms generate sample fields by means of stochastic relaxation implemented via the Gibbs s...

Absfruet-A new approach to regularization methods for image processing is introduced and developed using as a vehicle the problem of computing dense optical flow fields in an image sequence. Standard formulations of this problem require the computationally intensive solution of an elliptic partial differential equation that arises from the often used “smoothness constraint” ’yl”. regularization. The interpretation of the smoothness constraint is utilized as a “fractal prior ” to motivate regularization based on a recently introduced class of multiscale stochastic models. The solution of the new problem formulation is computed with an efficient multiscale algorithm. Experiments on several image sequences demonstrate the substantial computational savings that can be achieved due to the fact that the algorithm is noniterative and in fact has a per pixel computational complexity that is independent of image size. The new approach also has a number of other important advantages. Specifically, multiresolution flow field estimates are available, allowing great flexibility in dealing with the tradeoff between resolution and accuracy. Multiscale error covariance information is also available, which is of considerable use in assessing the accuracy of the estimates. In particular, these error statistics can be used as the basis for a rational procedure for determining the spatially-varying optimal reconstruction resolution. Furthermore, if there are compelling reasons to insist upon a standard smoothness constraint, our algorithm provides an excellent initialization for the iterative algorithms associated with the smoothness constraint problem formulation. Finally, the usefulness of our approach should extend to a wide variety of ill-posed inverse problems in which variational techniques seeking a “smooth ” solution are generally Used. I.

This paper describes a new motion estimation algorithm which is potentially useful for both computer vision and video compression applications. It is hierarchical in structure, using a separable 2-d Discrete Wavelet Transform (DWT) on each frame to efficiently construct a multiresolution pyramid of subimages. The DWT is based on a complex-valued pair of 4-tap FIR filters with Gabor-like characteristics. The resulting Complex DWT (CDWT) effectively implements an analysis by an ensemble of Gabor-like filters with a variety of orientations and scales. The phase difference between the subband coefficients of each frame at a given subpel bears a predictable relation to a local translation in the region of the reference frame subtended by that subpel. That relation is used to estimate the displacement field at the coarsest scale of the multiresolution pyramid. Each estimate is accompanied by a directional confidence measure in the form of the parameters of a quadratic matching surface. The i...

this paper should be helpful to researchers and practitioners working in the fields of video compression and processing, as well as in computer vision. Although the understanding of issues involved in the computation of motion has significantly increased over the last decade, we are still far from generic, robust, real-time motion estimation algorithms. The selection of the best motion estimator is still highly dependent on the application. Nevertheless, a broad variety of estimation models, criteria and optimization schemes can be treated in a unified framework presented here, thus allowing a direct comparison and leading to a deeper understanding of the properties of the resulting estimators.

Introduction In this chapter we are concerned with the estimation of 2-D motion from timevarying images and with the application of the computed motion to image sequence processing. Our goal for motion estimation is to propose a general formulation that incorporates object acceleration, nonlinear motion trajectories, occlusion effects and multichannel (vector) observations. To achieve this objective we use Gibbs-Markov models linked together by the Maximum A Posteriori Probability criterion which results in minimization of a multiple-term cost function. The specific applications of motion-compensated processing of image sequences are prediction, noise reduction and spatiotemporal interpolation. Estimation of motion from dynamic images is a very difficult task due to its ill-posedness [4]. Despite this difficulty, however, many approaches to the problem have been proposed in the last dozen years [27],[24],[40]. This activity can certainly be attrib

In this thesis, we develop image processing algorithms and applications for a particular class of multiscale stochastic models. First, we provide background on the model class, including a discussion of its relationship to wavelet transforms and the details of a two-sweep algorithm for estimation. A multiscale model for the error process associated with this algorithm is derived. Next, we illustrate how the multiscale models can be used in the context of regularizing ill-posed inverse problems and demonstrate the substantial computational savings that such an approach offers. Several novel features of the approach are developed including a technique for choosing the optimal resolution at which to recover the object of interest. Next, we show that this class of models contains other widely used classes of statistical models including 1-D Markov processes and 2-D Markov random fields, and we propose a class of multiscale models for approximately representing Gaussian Markov random fields...

This thesis addresses the problem of modeling and computing dense 2-D velocity and acceleration fields from time-varying images and applying them to motioncompensated interpolation. Unlike in many other approaches that assume motion to be locally translational, the approach proposed here uses a quadratic motion trajectory model that incorporates both velocity and acceleration. This model corresponds better to natural image sequences especially when processing over multiple frames is considered. One of the advantages of using accelerated motion over linear trajectories is in motion-compensated processing over multiple images. This is due to the fact that over longer time frame, a quadratic motion model is capable of providing a better intensity match along trajectories than the linear model. The side effect is, however, that with more images used for estimation occlusion effects play a more dominant role. Therefore, another motion model is proposed to account for occlusions and motion d...

This chapter presents a gradient-based approach to the multi-constraint estimation of dense two-dimensional (2-D) motion. The formulation is based on two assumptions: that at least one feature exists that is constant along motion trajectories and that motion vectors are smooth functions of spatial coordinates. From these assumptions matching and smoothness errors are derived and combined to obtain a cost function. The cost function is minimized using a sequence of quadratic approximations of the matching error and solving the resulting linear system by deterministic relaxation. The structural model used (relating the motion vectors and data) permits the use of multiple image features as the input, for example intensity and colours, or sub-bands of a spectral decomposition. The motion model reduces ill-posedness of the problem through a smoothness constraint. The proposed algorithm is a generalization of the Horn and Schunck algorithm [5] to the case of vector data. Results of applicat...

In this chapter we present Gibbs-Markov models for 2-D motion in the context of their application to video coding and processing. We study nonlinear trajectory model that incorporates both velocity and acceleration. Although the maximum a posteriori probability criterion is the preferred choice for most motion estimation algorithms based on Gibbs-Markov models, we discuss the more general Bayesian criterion, including the merits of several loss functions. We describe various models for the likelihood and prior probability distributions, but we concentrate on pixel-, block- and region-based motion models. We propose a new motion model that incorporates acceleration into the affine model. This contribution is mainly theoretical, however we present some experimental results to underline essential differences between models discussed. 4.1 Introduction In this chapter we are concerned with Gibbs-Markov models used in the computation of 2-D motion from time-varying images. Our goal is to pr...

This thesis is concerned primarily with the development of algorithms for estimating and segmenting image motion fields that contain discontinuities. An error-weighted regularization algorithm for image motion field estimation is proposed as a computationally attractive alternative to stochastic optimization based schemes. Block matching errors in the local motion measurement process are used in the regularization functional in order to avoid oversmoothing across motion boundaries. A second algorithm, anisotropic regularization, improves on the local measurement process, by employing alternative matching criteria and matching window organization. A selective confidence measure derived from anisotropic local measurements is used to further improve the error-weighted regularization.