We study an energy functional for computing optical flow that combines three assumptions: a brightness constancy assumption, a gradient constancy assumption, and a discontinuity-preserving spatio-temporal smoothness constraint.

We address the problem of vector-valued image regularization with variational methods and PDE's. From the study of existing formalisms, we propose a unifying framework based on a very local interpretation of the regularization processes. The resulting equations are then specialized into new regularization PDE's and corresponding numerical schemes that respect the local geometry of vector-valued images. They are finally applied on a wide variety of image processing problems, including color image restoration, inpainting, magnification and flow visualization.

AbstractÐIn this paper, we address the problem of estimating and analyzing the motion of fluids in image sequences. Due to the great deal of spatial and temporal distortions that intensity patterns exhibit in images of fluids, the standard techniques from Computer Vision, originally designed for quasi-rigid motions with stable salient features, are not well adapted in this context. We thus investigate a dedicated minimization-based motion estimator. The cost function to be minimized includes a novel data term relyingon an integrated version of the continuity equation of fluid mechanics, which is compatible with large displacements. This term is associated with an original second-order div-curl regularization which prevents the washing out of the salient vorticity and divergence structures. The performance of the resulting fluid flow estimator is demonstrated on meteorological satellite images. In addition, we show how the sequences of dense motion fields we estimate can be reliably used to reconstruct trajectories and to extract the regions of high vorticity and divergence. Index TermsÐFluid motion, continuity equation, div-curl regularization, nonconvex minimization, trajectories, vorticity, and divergence concentration. 1

Abstract not found

Abstract. We present a novel variational approach for segmenting the image plane into a set of regions of parametric motion on the basis of two consecutive frames from an image sequence. Our model is based on a conditional probability for the spatio-temporal image gradient, given a particular velocity model, and on a geometric prior on the estimated motion field favoring motion boundaries of minimal length. Exploiting the Bayesian framework, we derive a cost functional which depends on parametric motion models for each of a set of regions and on the boundary separating these regions. The resulting functional can be interpreted as an extension of the Mumford-Shah functional from intensity segmentation to motion segmentation. In contrast to most alternative approaches, the problems of segmentation and motion estimation are jointly solved by continuous minimization of a single functional. Minimizing this functional with respect to its dynamic variables results in an eigenvalue problem for the motion parameters and in a gradient descent evolution for the motion discontinuity set. We propose two different representations of this motion boundary: an explicit spline-based implementation which can be applied to the motion-based tracking of a single moving object, and an implicit multiphase level set implementation which allows for the segmentation of an arbitrary number of multiply connected moving objects. Numerical results both for simulated ground truth experiments and for real-world sequences demonstrate the capacity of our approach to segment objects based exclusively on their relative motion.

We are interested in regularizing fields of orthonormal vector sets, using constraint-preserving anisotropic diffusion PDE's. Each point of such a field is defined by multiple orthogonal and unitary vectors and can indeed represent a lot of interesting orientation features such as direction vectors or orthogonal matrices (among other examples). We first develop a general variational framework that solves this regularization problem, thanks to a constrained minimization of -functionals. This leads to a set of coupled vector-valued PDE's preserving the orthonormal constraints. Then, we focus on particular applications of this general framework, including the restoration of noisy direction fields, noisy chromaticity color images, estimated camera motions and DT-MRI (Di usion Tensor MRI) datasets.

Traditional techniques of dense optical flowestimation don't generally yield symmetrical solutions: the results will di#er if they are applied between images I1 and I2 or between images I2 and I1 . In this work, we present a method to recover a dense optical flow field map from two images, while explicitely taking into account the symmetry across the images as well as possible occlusions and discontinuities in the flow field. The idea is to consider both displacements vectors from I1 to I2 and I2 to I1 and to minimise an energy functional that explicitely encodes all those properties. This variational problem is then solved using the gradient flowdefined by theEuler--Lw7458) equations associated to the energy. In order to reduce the risk to be trapped within some irrelevant minimum, a focusing strategy based on a multi-resolution technique is used to converge toward the solution. Promising experimental results on both synthetic and real images are presented to illustrate the capabilities of this symmetrical variational approach to recover accurate optical flow. 1

Abstract. Variational methods are among the most accurate techniques for estimating the optic flow. They yield dense flow fields and can be designed such that they preserve discontinuities, estimate large displacements correctly and perform well under noise and varying illumination. However, such adaptations render the minimisation of the underlying energy functional very expensive in terms of computational costs: Typically one or more large linear or nonlinear equation systems have to be solved in order to obtain the desired solution. Consequently, variational methods are considered to be too slow for real-time performance. In our paper we address this problem in two ways: (i) We present a numerical framework based on bidirectional multigrid methods for accelerating a broad class of variational optic flow methods with different constancy and smoothness assumptions. Thereby, our work focuses particularly on regularisation strategies that preserve discontinuities. (ii) We show by the examples of five classical and two recent variational techniques that real-time performance is possible in all cases—even for very complex optic flow models that offer high accuracy. Experiments show that frame rates up to 63 dense flow fields per second for image sequences of size 160 × 120 can be achieved on a standard PC. Compared to classical iterative methods this constitutes a speedup of two to four orders of magnitude.

The goal of this paper is to present a unified description of diffusion and regularization techniques for vector-valued as well as matrix-valued data fields. In the vector-valued setting, we first review a number of existing methods and classify them into linear and nonlinear as well as isotropic and anisotropic methods. For these approaches we present corresponding regularization methods. This taxonomy is applied to the design of regularization methods for variational motion analysis in image sequences. Our vector-valued framework is then extended to the smoothing of positive semidefinite matrix fields. In this context a novel class of anisotropic di usion and regularization methods is derived and it is shown that suitable algorithmic realizations preserve the positive semidefiniteness of the matrix field without any additional constraints. As an application, we present an anisotropic nonlinear structure tensor and illustrate its advantages over the linear structure tensor.

We propose a variational method for segmenting image sequences into spatio-temporal domains of homogeneous motion. To this end, we formulate the problem of motion estimation in the framework of Bayesian inference, using a prior which favors domain boundaries of minimal surface area. We derive a cost functional which depends on a surface in space-time separating a set of motion regions, as well as a set of vectors modeling the motion in each region. We propose a multiphase level set formulation of this functional, in which the surface and the motion regions are represented implicitly by a vector-valued level set function. Joint minimization of the proposed functional results in an eigenvalue problem for the motion model of each region and in a gradient descent evolution for the separating interface. Numerical results on real-world sequences demonstrate that minimization of a single cost functional generates a segmentation of space-time into multiple motion regions.