## A multigrid platform for real-time motion computation with discontinuity-preserving variational methods (2006)

### Cached

### Download Links

- [www.cvgpr.uni-mannheim.de]
- [ipa.iwr.uni-heidelberg.de]
- [www.mia.uni-saarland.de]
- [www.math.uni-sb.de]
- [www.math.uni-sb.de]
- DBLP

### Other Repositories/Bibliography

Venue: | International Journal of Computer Vision |

Citations: | 41 - 12 self |

### BibTeX

@ARTICLE{Bruhn06amultigrid,

author = {Andrés Bruhn and Joachim Weickert},

title = {A multigrid platform for real-time motion computation with discontinuity-preserving variational methods},

journal = {International Journal of Computer Vision},

year = {2006},

volume = {70},

pages = {257--277}

}

### OpenURL

### Abstract

Abstract. Variational methods are among the most accurate techniques for estimating the optic flow. They yield dense flow fields and can be designed such that they preserve discontinuities, estimate large displacements correctly and perform well under noise and varying illumination. However, such adaptations render the minimisation of the underlying energy functional very expensive in terms of computational costs: Typically one or more large linear or nonlinear equation systems have to be solved in order to obtain the desired solution. Consequently, variational methods are considered to be too slow for real-time performance. In our paper we address this problem in two ways: (i) We present a numerical framework based on bidirectional multigrid methods for accelerating a broad class of variational optic flow methods with different constancy and smoothness assumptions. Thereby, our work focuses particularly on regularisation strategies that preserve discontinuities. (ii) We show by the examples of five classical and two recent variational techniques that real-time performance is possible in all cases—even for very complex optic flow models that offer high accuracy. Experiments show that frame rates up to 63 dense flow fields per second for image sequences of size 160 × 120 can be achieved on a standard PC. Compared to classical iterative methods this constitutes a speedup of two to four orders of magnitude.

### Citations

1826 |
Robust statistics
- Huber
- 1981
(Show Context)
Citation Context ... than in the quadratic setting (L2 norm). As a consequence, large gradient features such as edges are better preserved. Such a form of penalisation can be related to statistically robust error norms (=-=Huber, 1981-=-). The corresponding energy functional reads � � EFI(u,v) = (u,v,1)J (u,v,1) � ⊤ � � � 2 2 + α�S |∇u| +|∇v| dxdy, (9) where �s(s 2 ) is a positive increasing function in R with the property to increas... |

1734 | Determining optical flow
- Horn, Schunck
- 1981
(Show Context)
Citation Context ...rategies to regularise this often non-unique solution of a data term: homogeneous regularisation that assumes overall smoothness and does not adapt to semantically important image or flow structures (=-=Horn and Schunck, 1981-=-) image-driven regularisation that assumes piecewise smoothness and respects discontinuities in the image (Alvarez et al., 1999a; Nagel and Enkelmann, 1986) flow-driven regularisation that assumes pie... |

1370 |
Nonlinear total variation based noise removal algorithms
- Rudin, Osher, et al.
- 1992
(Show Context)
Citation Context ... less severely than a quadratic function. As prototype we have chosen a method that penalises deviations from the smoothness with the L1 norm. This corresponds to total variation (TV) regularisation (=-=Rudin et al., 1992-=-) which we implemented by means of a regularised variant given by �S(s 2 ) = � s2 + ɛ2 S . (10) Here, ɛS serves as small regularisation parameter. A similar functional that approximates TV regularisat... |

1047 | Performance of optical flow techniques
- Barron, Fleet, et al.
- 1994
(Show Context)
Citation Context ...developments (Brox et al., 2004; Bruhn et al., 2005c; Mémin and Pérez, 1998) have shown that variational methods are among the best techniques for computing the optic flow in terms of error measures (=-=Barron et al., 1994-=-), they are often considered to be too slow for real-time applications. In particular, the computational costs for solving the resulting linear and nonlinear systems of equations are regarded as too h... |

822 |
Solution of Ill-posed Problems
- Tikhonov, Arsenin
- 1977
(Show Context)
Citation Context ...lass of methods with homogeneous regularisation is the classical method of Horn and Schunck (1981). Their method assumes global smoothness by penalising deviations from smoothness in a quadratic way (=-=Tikhonov and Arsenin, 1977-=-). The corresponding energy functional reads � � ⊤ EHOM(u,v) = (u,v,1)J(u,v,1) � +α � |∇u| 2 +|∇v| 2 �� dxdy, (4) where the regularisation parameter α is a positive number that steers the smoothness o... |

761 |
Multi-grid methods and applications
- Hackbusch
- 1985
(Show Context)
Citation Context ...nal multigrid schemes are not the end of the road. Very promising—in particular in the context of variational methods—are so-called bidirectional multigrid methods (Brandt, 1977; Briggs et al., 2000; =-=Hackbusch, 1985-=-; Trottenberg et al., 2001; Wesseling, 1992). These techniques that create a sophisticated hierarchy of equation systems with excellent error reduction properties belong to the fastest numerical schem... |

578 | Heirarchical model-based motion estimation
- Bergen, Anandan, et al.
- 1992
(Show Context)
Citation Context ...n be avoided by a coarse-to-fine framework. In the nonconvex case—in particular in the context of large displacements—unidirectional multigrid schemes are often combined with so-called warping steps (=-=Bergen et al., 1992-=-; Black and Anandan, 1991; Mémin and Pérez, 1998; Brox et al., 2004). Thereby the original problem is compensated by the already computed motion from all coarser levels before the resolution is refine... |

531 | The robust estimation of multiple motions: Parametric and piecewise-smooth flow fields - Black, Anandan - 1996 |

514 |
A Multigrid Tutorial
- Briggs, Henson, et al.
- 2000
(Show Context)
Citation Context ...ions by using standard iterative methods are regarded as too high. In [10] we have already demonstrated for variational methods with homogeneous regularisation that bidirectional multigrid strategies =-=[7, 8, 26, 42, 48]-=- do allow for real-time performance. These techniques that create a sophisticated hierarchy of equation systems with excellent error reduction properties belong to the fastest numerical schemes for so... |

509 |
Iterative Solutions of Nonlinear Equations
- Ortega, Rheinboldt
(Show Context)
Citation Context ...d 5.1. Basic Concept In general, the preceding linear and nonlinear systems of equations are solved by using non-hierarchical iterative schemes; e.g. variants of the Jacobi or the Gauß-Seidel method (=-=Ortega and Rheinboldt, 2000-=-; Young, 1971). However, such techniques are not well–suited for equation systems that are only coupled via a small local neighbourhood: It may take thousands of iterations to transport local informat... |

451 | A computational framework and an algorithm for the measurement of visual motion - Anandan - 1989 |

354 |
Multi-level adaptive solutions to boundary-value problems
- Brandt
- 1977
(Show Context)
Citation Context ...erical viewpoint, these unidirectional multigrid schemes are not the end of the road. Very promising—in particular in the context of variational methods—are so-called bidirectional multigrid methods (=-=Brandt, 1977-=-; Briggs et al., 2000; Hackbusch, 1985; Trottenberg et al., 2001; Wesseling, 1992). These techniques that create a sophisticated hierarchy of equation systems with excellent error reduction properties... |

341 |
An Introduction to Multigrid Methods
- Wesseling
- 1992
(Show Context)
Citation Context ...e road. Very promising—in particular in the context of variational methods—are so-called bidirectional multigrid methods (Brandt, 1977; Briggs et al., 2000; Hackbusch, 1985; Trottenberg et al., 2001; =-=Wesseling, 1992-=-). These techniques that create a sophisticated hierarchy of equation systems with excellent error reduction properties belong to the fastest numerical schemes for solving linear or nonlinear systems ... |

335 | Iterative Solution of Large Linear Systems - Young - 1971 |

299 | High accuracy optical flow estimation based on a theory for warping
- Brox, Bruhn, et al.
- 2004
(Show Context)
Citation Context ... value of objects, and a smoothness term that regularises the often non-unique (local)s258 Bruhn et al. solution of the data term by an additional smoothness assumption. Although recent developments (=-=Brox et al., 2004-=-; Bruhn et al., 2005c; Mémin and Pérez, 1998) have shown that variational methods are among the best techniques for computing the optic flow in terms of error measures (Barron et al., 1994), they are ... |

221 |
Anisotropic Diffusion
- Weickert
- 1998
(Show Context)
Citation Context ... to solving their Euler– Lagrange equations. As for the motion tensor in the data term, also a very compact and general formulation for the smoothness term is possible: the diffusion tensor notation (=-=Weickert, 1998-=-). Let us now explain this notation by the example of the Euler-Lagrange equations (a)–(e). 2.3.1. The Diffusion Tensor Notation (a)–(c) The Linear Case In the first three cases (a), (b) and (c) the E... |

216 |
Enkelmann, An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences
- Nagel, W
- 1986
(Show Context)
Citation Context ...emantically important image or flow structures (Horn and Schunck, 1981) image-driven regularisation that assumes piecewise smoothness and respects discontinuities in the image (Alvarez et al., 1999a; =-=Nagel and Enkelmann, 1986-=-) flow-driven regularisation that assumes piecewise smoothness and respects discontinuities in the flow field; see e.g. (Cohen, 1993; Schnörr, 1994; Weickert and Schnörr, 2001a). Moreover, when consid... |

207 |
A fast operator for detection and precise location of distinct points, corners and centres of circular features
- Förstner, Gülch
- 1987
(Show Context)
Citation Context ..., 2005). However, in the case of the grey value constancy assumption, the obtained quadratic form is very special: The associated motion tensor coincides exactly with the well-known structure tensor (=-=Förstner and Gülch, 1987-=-). 2.2. The Smoothness Term Obviously, in case of a singular motion tensor, the solution of equation 3 is non-unique. Variational methods overcome this so–called aperture problem by additionally assum... |

197 |
Computational Methods for Inverse Problems
- Vogel
(Show Context)
Citation Context ...er we used the strategy of frozen coefficients (Frohn-Schnauff et al., 2004). In the literature this technique is also known as lagged diffusivity method (Chan and Mulet, 1999) or QuasiNewton scheme (=-=Vogel, 2002-=-). Direct nonlinear Gauß– Seidel Newton methods (Briggs et al., 2000) have not been considered as basic solver. Experiments using this kind of methods have shown a similar performance in terms of erro... |

140 | Lucas/Kanade meets Horn/Schunck: Combining local and global optic flow methods - Weickert, Bruhn, et al. - 2003 |

107 | Robust dynamic motion estimation over time
- Black, Anandan
- 1991
(Show Context)
Citation Context ...general good approximations to fine grid solutions. On the other hand–in the case of nonconvex energy functionals–they allow to improve the quality of the results significantly (Alvarez et al., 2000; =-=Black and Anandan, 1991-=-; Bruhn et al., 2005c; Mémin and Pérez, 1998; Papenberg et al., 2005). This is a direct consequence of the fact that certain undesired local minima disappear at sufficiently coarse resolutions and thu... |

97 | Reliable estimation of dense optical flow fields with large displacements
- Alvarez, Weickert, et al.
- 2000
(Show Context)
Citation Context ...e grid results are in general good approximations to fine grid solutions. On the other hand–in the case of nonconvex energy functionals–they allow to improve the quality of the results significantly (=-=Alvarez et al., 2000-=-; Black and Anandan, 1991; Bruhn et al., 2005c; Mémin and Pérez, 1998; Papenberg et al., 2005). This is a direct consequence of the fact that certain undesired local minima disappear at sufficiently c... |

97 | Efficient multiscale regularization with applications to the computation of optical flow - Luettgen, Karl, et al. - 1994 |

97 | Dense estimation and object-based segmentation of the optical flow with robust techniques
- Mémin, Pérez
- 1998
(Show Context)
Citation Context ... that regularises the often non-unique (local)s258 Bruhn et al. solution of the data term by an additional smoothness assumption. Although recent developments (Brox et al., 2004; Bruhn et al., 2005c; =-=Mémin and Pérez, 1998-=-) have shown that variational methods are among the best techniques for computing the optic flow in terms of error measures (Barron et al., 1994), they are often considered to be too slow for real-tim... |

89 |
Investigations of multigrid algorithms for the estimation of optical flow fields in image sequences
- Enkelmann
- 1988
(Show Context)
Citation Context ...ately, these approaches are generally restricted to linear techniques with homogeneous and image-driven regularisation (Glazer, 1984; Terzopoulos, 1986; Zini et al., 1997; El Kalmoun and Rüde, 2003; =-=Enkelmann, 1987-=-; Ghosal and Vaněk, 1996). Only the work of (Borzi et al., 2002) is known to the authors where a nonlinear optic flow problem with flow-driven regularisation was solved by means of a suitable bidirect... |

78 | Highly accurate optic flow computation with theoretically justified warping
- Papenberg, Bruhn, et al.
- 2006
(Show Context)
Citation Context ...–in the case of nonconvex energy functionals–they allow to improve the quality of the results significantly (Alvarez et al., 2000; Black and Anandan, 1991; Bruhn et al., 2005c; Mémin and Pérez, 1998; =-=Papenberg et al., 2005-=-). This is a direct consequence of the fact that certain undesired local minima disappear at sufficiently coarse resolutions and thus can be avoided by a coarse-to-fine framework. In the nonconvex cas... |

77 | A theoretical framework for convex regularizers in PDE–based computation of image motion - Weickert, Schnörr |

75 | Two deterministic half-quadratic regularization algorithms for computed imaging - Charbonnier, Blanc-F'eraud, et al. - 1994 |

74 | Variational Optic flow computation with a spatio-temporal smoothness constraint - Weickert, Schnorr - 2001 |

43 | On the convergence of the lagged diffusivity fixed point method in total variation image restoration
- Chan, Mulet
- 1999
(Show Context)
Citation Context ...he nonlinear variants of the Gauß–Seidel solver we used the strategy of frozen coefficients (Frohn-Schnauff et al., 2004). In the literature this technique is also known as lagged diffusivity method (=-=Chan and Mulet, 1999-=-) or QuasiNewton scheme (Vogel, 2002). Direct nonlinear Gauß– Seidel Newton methods (Briggs et al., 2000) have not been considered as basic solver. Experiments using this kind of methods have shown a ... |

42 | Very high accuracy velocity estimation using orientation tensors, parametric motion, and simultaneous segmentation of the motion field - Farneback - 2001 |

40 |
The cascadic multigrid method for elliptic problems
- Bornemann, Deuflhard
- 1996
(Show Context)
Citation Context ... relaxation (ALR) and the Gauß-Seidel method with coupled point relaxation (CPR). Moreover, we considered unidirectional multigrid variants of the basics solvers—so-called cascadic multigrid methods (=-=Bornemann and Deuflhard, 1996-=-)—as well as modified explicit schemes (Weickert and Schnörr, 2001). Compared to ordinary explicit schemes (e.g. gradient descent methods) such modified schemes allow for larger time step sizes τ. For... |

38 | A multigrid approach for hierarchical motion estimation
- Mémin, Pérez
- 1998
(Show Context)
Citation Context ... that regularises the often non-unique (local)s258 Bruhn et al. solution of the data term by an additional smoothness assumption. Although recent developments (Brox et al., 2004; Bruhn et al., 2005c; =-=Mémin and Pérez, 1998-=-) have shown that variational methods are among the best techniques for computing the optic flow in terms of error measures (Barron et al., 1994), they are often considered to be too slow for real-tim... |

37 | A scale-space approach to nonlocal optical flow calculations
- Alvarez, Weickert, et al.
- 1999
(Show Context)
Citation Context ...and does not adapt to semantically important image or flow structures (Horn and Schunck, 1981) image-driven regularisation that assumes piecewise smoothness and respects discontinuities in the image (=-=Alvarez et al., 1999-=-a; Nagel and Enkelmann, 1986) flow-driven regularisation that assumes piecewise smoothness and respects discontinuities in the flow field; see e.g. (Cohen, 1993; Schnörr, 1994; Weickert and Schnörr, 2... |

35 | Towards ultimate motion estimation: Combining highest accuracy with real-time performance - Bruhn, Weickert - 2005 |

31 |
A fast scalable algorithm for discontinuous optical flow estimation
- Ghosal, Vanek
- 1996
(Show Context)
Citation Context ...oaches are generally restricted to linear techniques with homogeneous and image-driven regularisation (Glazer, 1984; Terzopoulos, 1986; Zini et al., 1997; El Kalmoun and Rüde, 2003; Enkelmann, 1987; =-=Ghosal and Vaněk, 1996-=-). Only the work of (Borzi et al., 2002) is known to the authors where a nonlinear optic flow problem with flow-driven regularisation was solved by means of a suitable bidirectional multigrid scheme (... |

30 | Nonlinear variational method for optical flow computation
- Cohen
- 1993
(Show Context)
Citation Context ... discontinuities in the image (Alvarez et al., 1999a; Nagel and Enkelmann, 1986) flow-driven regularisation that assumes piecewise smoothness and respects discontinuities in the flow field; see e.g. (=-=Cohen, 1993-=-; Schnörr, 1994; Weickert and Schnörr, 2001a). Moreover, when considering image and flow-driven regularisation, one can distinguish between isotropic and anisotropic smoothness terms. While isotropic ... |

29 | Optical-Flow Estimation while Preserving Its Discontinuities: A Variational Approach - Deriche, Kornprobst, et al. - 1995 |

25 |
Multilevel relaxation in low-level computer vision. Multi-resolution image processing and Analysis
- Glazer
- 1984
(Show Context)
Citation Context ... of such methods in the context of variational optic flow computation. Unfortunately, these approaches are generally restricted to linear techniques with homogeneous and image-driven regularisation (=-=Glazer, 1984-=-; Terzopoulos, 1986; Zini et al., 1997; El Kalmoun and Rüde, 2003; Enkelmann, 1987; Ghosal and Vaněk, 1996). Only the work of (Borzi et al., 2002) is known to the authors where a nonlinear optic flow ... |

24 |
A PDE model for computing the optical flow
- Alvarez, Esclarín, et al.
- 1999
(Show Context)
Citation Context ...and does not adapt to semantically important image or flow structures (Horn and Schunck, 1981) image-driven regularisation that assumes piecewise smoothness and respects discontinuities in the image (=-=Alvarez et al., 1999-=-a; Nagel and Enkelmann, 1986) flow-driven regularisation that assumes piecewise smoothness and respects discontinuities in the flow field; see e.g. (Cohen, 1993; Schnörr, 1994; Weickert and Schnörr, 2... |

24 | I.: An algebraic multigrid approach for image analysis
- Kimmel, Yavneh
- 2003
(Show Context)
Citation Context ...d scheme (FAS). Also for other tasks in image processing and computer vision, bidirectional multigrid methods have been used successfully. In the context of photometric stereo and image binarisation (=-=Kimmel and Yavneh, 2003-=-) developed an algebraic multigrid method, while (Chan et al., 1997) researched geometric multigrid schemes for variational deconvolution with total variation (TV) regularisation. For TV denoising (Vo... |

21 |
Calculus of Variations
- Elsgolc
- 1962
(Show Context)
Citation Context .... As for the isotropic case we have chosen a method as prototype that is based on a regularised variant of the total variation. 2.3. The Euler–Lagrange Equations Following the calculus of variations (=-=Elsgolc, 1961-=-), the minimisation of the previously discussed energy functionals comes down to solving their Euler– Lagrange equations. As for the motion tensor in the data term, also a very compact and general for... |

19 | Real-Time Optic Flow Computation with Variational Methods - Bruhn, Weickert, et al. - 2004 |

19 | Optical flow: A curve evolution approach - Kumar, Tannenbaum, et al. - 1996 |

17 |
Optimal control formulation for determining optical flow
- Borz, Ito, et al.
- 2002
(Show Context)
Citation Context ...echniques with homogeneous and image-driven regularisation (Glazer, 1984; Terzopoulos, 1986; Zini et al., 1997; El Kalmoun and Rüde, 2003; Enkelmann, 1987; Ghosal and Vaněk, 1996). Only the work of (=-=Borzi et al., 2002-=-) is known to the authors where a nonlinear optic flow problem with flow-driven regularisation was solved by means of a suitable bidirectional multigrid scheme (FAS). Also for other tasks in image pro... |

17 | Multigrid for differential-convolution problems arising in image processing
- Chan, Chan, et al.
- 1999
(Show Context)
Citation Context ...ion, bidirectional multigrid methods have been used successfully. In the context of photometric stereo and image binarisation (Kimmel and Yavneh, 2003) developed an algebraic multigrid method, while (=-=Chan et al., 1997-=-) researched geometric multigrid schemes for variational deconvolution with total variation (TV) regularisation. For TV denoising (Vogel, 1995) proposed the use of a linear multigrid method within a n... |

14 | A multigrid method for total variation-based image denoising
- Vogel
- 1995
(Show Context)
Citation Context ...03) developed an algebraic multigrid method, while (Chan et al., 1997) researched geometric multigrid schemes for variational deconvolution with total variation (TV) regularisation. For TV denoising (=-=Vogel, 1995-=-) proposed the use of a linear multigrid method within a nonlinear fixed-point iteration, while, very recently, Frohn-Schnauf et al. (2004) investigated a nonlinear multigrid scheme (FAS) for the same... |

13 | Variational optic flow computation in real-time
- Bruhn, Weickert, et al.
- 2005
(Show Context)
Citation Context ...ar and nonlinear cases. Thereby we used two different types of basic solvers: While in the cases of homogeneous and isotropic regularisation, a Gauß–Seidel solver with coupled point relaxation (CPR) (=-=Bruhn et al., 2005a-=-) was sufficient, the anisotropy of the neighbourhood coupling in the remaining methods required the use of a Gauß– Seidel solver with alternating line relaxation (ALR) (Wesseling, 1992). Instead of u... |

12 | Nonlinear multigrid methods for total variation image denoising - Frohn-Schauf, Henn, et al. - 2004 |

7 | U.: A variational multigrid for computing the optical flow. Vision, Modeling and Visualization 2003 (Berlin 2003) 577–584 - Kalmoun, Rüde |