Results 1  10
of
109
A database and evaluation methodology for optical flow
 In Proceedings of the IEEE International Conference on Computer Vision
, 2007
"... The quantitative evaluation of optical flow algorithms by Barron et al. (1994) led to significant advances in performance. The challenges for optical flow algorithms today go beyond the datasets and evaluation methods proposed in that paper. Instead, they center on problems associated with complex n ..."
Abstract

Cited by 404 (21 self)
 Add to MetaCart
(Show Context)
The quantitative evaluation of optical flow algorithms by Barron et al. (1994) led to significant advances in performance. The challenges for optical flow algorithms today go beyond the datasets and evaluation methods proposed in that paper. Instead, they center on problems associated with complex natural scenes, including nonrigid motion, real sensor noise, and motion discontinuities. We propose a new set of benchmarks and evaluation methods for the next generation of optical flow algorithms. To that end, we contribute four types of data to test different aspects of optical flow algorithms: (1) sequences with nonrigid motion where the groundtruth flow is determined by tracking hidden fluorescent texture, (2) realistic synthetic sequences, (3) high framerate video used to study interpolation error, and (4) modified stereo sequences of static scenes. In addition to the average angular error used by Barron et al., we compute the absolute flow endpoint error, measures for frame interpolation error, improved statistics, and results at motion discontinuities and in textureless regions. In October 2007, we published the performance of several wellknown methods on a preliminary version of our data to establish the current state of the art. We also made the data freely available on the web at
A duality based approach for realtime tvl1 optical flow
 In Ann. Symp. German Association Patt. Recogn
, 2007
"... Abstract. Variational methods are among the most successful approaches to calculate the optical flow between two image frames. A particularly appealing formulation is based on total variation (TV) regularization and the robust L 1 norm in the data fidelity term. This formulation can preserve discont ..."
Abstract

Cited by 197 (15 self)
 Add to MetaCart
(Show Context)
Abstract. Variational methods are among the most successful approaches to calculate the optical flow between two image frames. A particularly appealing formulation is based on total variation (TV) regularization and the robust L 1 norm in the data fidelity term. This formulation can preserve discontinuities in the flow field and offers an increased robustness against illumination changes, occlusions and noise. In this work we present a novel approach to solve the TVL 1 formulation. Our method results in a very efficient numerical scheme, which is based on a dual formulation of the TV energy and employs an efficient pointwise thresholding step. Additionally, our approach can be accelerated by modern graphics processing units. We demonstrate the realtime performance (30 fps) of our approach for video inputs at a resolution of 320 × 240 pixels. 1
Nonlocal image and movie denoising
 International Journal of Computer Vision
, 2008
"... Neighborhood filters are nonlocal image and movie filters which reduce the noise by averaging similar pixels. The first object of the paper is to present a unified theory of these filters and reliable criteria to compare them to other filter classes. A CCD noise model will be presented justifying th ..."
Abstract

Cited by 99 (2 self)
 Add to MetaCart
(Show Context)
Neighborhood filters are nonlocal image and movie filters which reduce the noise by averaging similar pixels. The first object of the paper is to present a unified theory of these filters and reliable criteria to compare them to other filter classes. A CCD noise model will be presented justifying the involvement of neighborhood filters. A classification of neighborhood filters will be proposed, including classical image and movie denoising methods and discussing further a recently introduced neighborhood filter, NLmeans. In order to compare denoising methods three principles will be discussed. The first principle, “method noise”, specifies that only noise must be removed from an image. A second principle will be introduced, “noise to noise”, according to which a denoising method must transform a white noise into a white noise. Contrarily to “method noise”, this principle, which characterizes artifactfree methods, eliminates any subjectivity and can be checked by mathematical arguments and Fourier analysis. “Noise to noise ” will be proven to rule out most denoising methods, with the exception of neighborhood filters. This is why a third and new comparison principle, the “statistical optimality”, is needed and will be
Fusion Moves for Markov Random Field Optimization
"... The efficient application of graph cuts to Markov Random Fields (MRFs) with multiple discrete or continuous labels remains an open question. In this paper, we demonstrate one possible way of achieving this by using graph cuts to combine pairs of suboptimal labelings or solutions. We call this combi ..."
Abstract

Cited by 67 (5 self)
 Add to MetaCart
(Show Context)
The efficient application of graph cuts to Markov Random Fields (MRFs) with multiple discrete or continuous labels remains an open question. In this paper, we demonstrate one possible way of achieving this by using graph cuts to combine pairs of suboptimal labelings or solutions. We call this combination process the fusion move. By employing recently developed graph cut based algorithms (socalled QPBOgraph cut), the fusion move can efficiently combine two proposal labelings in a theoretically sound way, which is in practice often globally optimal. We demonstrate that fusion moves generalize many previous graph cut approaches, which allows them to be used as building block within a broader variety of optimization schemes than were considered before. In particular, we propose new optimization schemes for computer vision MRFs with applications to image restoration, stereo, and optical flow, among others. Within these schemes the fusion moves are used 1) for the parallelization of MRF optimization into several threads; 2) for fast MRF optimization by combining cheaptocompute solutions; and 3) for the optimization of highly nonconvex continuouslabeled MRFs with 2D labels. Our final example is a nonvision MRF concerned with cartographic label placement, where fusion moves can be used to improve the performance of a standard inference method (loopy belief propagation).
A multigrid platform for realtime motion computation with discontinuitypreserving variational methods
 International Journal of Computer Vision
, 2006
"... Abstract. Variational methods are among the most accurate techniques for estimating the optic flow. They yield dense flow fields and can be designed such that they preserve discontinuities, estimate large displacements correctly and perform well under noise and varying illumination. However, such ad ..."
Abstract

Cited by 58 (16 self)
 Add to MetaCart
(Show Context)
Abstract. Variational methods are among the most accurate techniques for estimating the optic flow. They yield dense flow fields and can be designed such that they preserve discontinuities, estimate large displacements correctly and perform well under noise and varying illumination. However, such adaptations render the minimisation of the underlying energy functional very expensive in terms of computational costs: Typically one or more large linear or nonlinear equation systems have to be solved in order to obtain the desired solution. Consequently, variational methods are considered to be too slow for realtime performance. In our paper we address this problem in two ways: (i) We present a numerical framework based on bidirectional multigrid methods for accelerating a broad class of variational optic flow methods with different constancy and smoothness assumptions. Thereby, our work focuses particularly on regularisation strategies that preserve discontinuities. (ii) We show by the examples of five classical and two recent variational techniques that realtime performance is possible in all cases—even for very complex optic flow models that offer high accuracy. Experiments show that frame rates up to 63 dense flow fields per second for image sequences of size 160 × 120 can be achieved on a standard PC. Compared to classical iterative methods this constitutes a speedup of two to four orders of magnitude.
FusionFlow: DiscreteContinuous Optimization for Optical Flow Estimation
, 2008
"... Accurate estimation of optical flow is a challenging task, which often requires addressing difficult energy optimization problems. To solve them, most topperforming methods rely on continuous optimization algorithms. The modeling accuracy of the energy in this case is often traded for its tractabil ..."
Abstract

Cited by 58 (7 self)
 Add to MetaCart
(Show Context)
Accurate estimation of optical flow is a challenging task, which often requires addressing difficult energy optimization problems. To solve them, most topperforming methods rely on continuous optimization algorithms. The modeling accuracy of the energy in this case is often traded for its tractability. This is in contrast to the related problem of narrowbaseline stereo matching, where the topperforming methods employ powerful discrete optimization algorithms such as graph cuts and messagepassing to optimize highly nonconvex energies. In this paper, we demonstrate how similar nonconvex energies can be formulated and optimized discretely in the context of optical flow estimation. Starting with a set of candidate solutions that are produced by fast continuous flow estimation algorithms, the proposed method iteratively fuses these candidate solutions by the computation of minimum cuts on graphs. The obtained continuousvalued fusion result is then further improved using local gradient descent. Experimentally, we demonstrate that the proposed energy is an accurate model and that the proposed discretecontinuous optimization scheme not only finds lower energy solutions than traditional discrete or continuous optimization techniques, but also leads to flow estimates that outperform the current stateoftheart.
An improved algorithm for TVL1 optical flow
 In: Visual Motion Analysis Workshop, LNCS 5604
, 2009
"... Fig. 1. Optical flow for the backyard and mini cooper scene of the Middlebury optical flow benchmark. Optical flow captures the dynamics of a scene by estimating the motion of every pixel between two frames of an image sequence. The displacement of every pixel is shown as displacement vectors on top ..."
Abstract

Cited by 52 (5 self)
 Add to MetaCart
(Show Context)
Fig. 1. Optical flow for the backyard and mini cooper scene of the Middlebury optical flow benchmark. Optical flow captures the dynamics of a scene by estimating the motion of every pixel between two frames of an image sequence. The displacement of every pixel is shown as displacement vectors on top of the commonly used flow color scheme (see Figure 5). Abstract. A look at the Middlebury optical flow benchmark [5] reveals that nowadays variational methods yield the most accurate optical flow fields between two image frames. In this work we propose an improvement variant of the original duality based TVL 1 optical flow algorithm in [31] and provide implementation details. This formulation can preserve discontinuities in the flow field by employing total variation (TV) regularization. Furthermore, it offers robustness against outliers by applying the robust L 1 norm in the data fidelity term. Our contributions are as follows. First, we propose to perform a structuretexture decomposition of the input images to get rid of violations in the optical flow constraint due to illumination changes. Second, we propose to integrate a median filter into the numerical scheme to further increase the robustness to sampling artefacts in the image data. We experimentally show that very precise and robust estimation of optical flow can be achieved with a variational approach in realtime. The numerical scheme and the implementation are described in a detailed way, which enables reimplementation of this highend method. 2 A. Wedel, T. Pock, C. Zach, H. Bischof, and D. Cremers 1
Variational motion segmentation with level sets
 in ECCV
, 2006
"... Abstract. We suggest a variational method for the joint estimation of optic flow and the segmentation of the image into regions of similar motion. It makes use of the level set framework following the idea of motion competition, which is extended to nonparametric motion. Moreover, we automatically ..."
Abstract

Cited by 41 (4 self)
 Add to MetaCart
(Show Context)
Abstract. We suggest a variational method for the joint estimation of optic flow and the segmentation of the image into regions of similar motion. It makes use of the level set framework following the idea of motion competition, which is extended to nonparametric motion. Moreover, we automatically determine an appropriate initialization and the number of regions by means of recursive twophase splits with higher order region models. The method is further extended to the spatiotemporal setting and the use of additional cues like the gray value or color for the segmentation. It need not fear a quantitative comparison to pure optic flow estimation techniques: For the popular Yosemite sequence with clouds we obtain the currently most accurate result. We further uncover a mistake in the ground truth. Coarsely correcting this, we get an average angular error below 1 degree. 1
Overparameterized variational optical flow
 International Journal of Computer Vision
"... We introduce a novel optical flow estimation process based on a spatiotemporal model with varying coefficients multiplying a set of basis functions at each pixel. Previous optical flow estimation methodologies did not use such an over parameterized representation of the flow field as the problem is ..."
Abstract

Cited by 33 (1 self)
 Add to MetaCart
(Show Context)
We introduce a novel optical flow estimation process based on a spatiotemporal model with varying coefficients multiplying a set of basis functions at each pixel. Previous optical flow estimation methodologies did not use such an over parameterized representation of the flow field as the problem is illposed even without introducing any additional parameters: Neighborhood based methods like LucasKanade determine the flow in each pixel by constraining the flow to to be constant in a small area. Modern variational methods represent the optic flow directly via its x and y components at each pixel. The benefit of overparametrization becomes evident in the smoothness term, which instead of directly penalizing for changes in the optic flow, integrates a cost on the deviation from the assumed optic flow model. Previous variational optical flow techniques are special cases of the proposed method, used in conjunction with a constant flow basis function. Experimental results with the novel flow estimation process yielded significant improvements with respect to the best results published so far. 1.