The problem of tracking curves in dense visual clutter is challenging. Kalman filtering is inadequate because it is based on Gaussian densities which, being unimodal, cannot represent simultaneous alternative hypotheses. The Condensation algorithm uses "factored sampling", previously applied to the interpretation of static images, in which the probability distribution of possible interpretations is represented by a randomly generated set. Condensation uses learned dynamical models, together with visual observations, to propagate the random set over time. The result is highly robust tracking of agile motion. Notwithstanding the use of stochastic methods, the algorithm runs in near real-time. Contents 1 Tracking curves in clutter 2 2 Discrete-time propagation of state density 3 3 Factored sampling 6 4 The Condensation algorithm 8 5 Stochastic dynamical models for curve motion 10 6 Observation model 13 7 Applying the Condensation algorithm to video-streams 17 8 Conclusions 26 A Non-line...

. In Proc. European Conf. Computer Vision, 1996, pp. 343--356, Cambridge, UK The problem of tracking curves in dense visual clutter is a challenging one. Trackers based on Kalman filters are of limited use; because they are based on Gaussian densities which are unimodal, they cannot represent simultaneous alternative hypotheses. Extensions to the Kalman filter to handle multiple data associations work satisfactorily in the simple case of point targets, but do not extend naturally to continuous curves. A new, stochastic algorithm is proposed here, the Condensation algorithm --- Conditional Density Propagation over time. It uses `factored sampling', a method previously applied to interpretation of static images, in which the distribution of possible interpretations is represented by a randomly generated set of representatives. The Condensation algorithm combines factored sampling with learned dynamical models to propagate an entire probability distribution for object pos...

This thesis establishes a stochastic framework for tracking curves in visual clutter, using a Bayesian random-sampling algorithm. The approach is rooted in ideas from statistics, control theory and computer vision. The problem is to track outlines and features of foreground objects, modelled as curves, as they move in substantial clutter, and to do it at, or close to, video frame-rate. The algorithm, named Condensation, for Conditional density propagation, has recently been derived independently by several researchers, and is generating signi cant interest in the statistics and signal processing communities. This thesis contributes to the literature on Condensation-like lters by presenting some novel applications of and extensions to the basic algorithm, and contributes to the visual motion estimation literature by demonstrating high tracking performance in cluttered environments. Despite its power the Condensation algorithm has a remarkably simple form and this allows the use of non-linear motion models which combine characteristics of discrete Hidden Markov Models with the continuous Auto-Regressive Process motion models traditionally used in Kalman lters. These mixed discrete-continuous models have promising applications to the emerging eld of perception of action. This thesis also implements two algorithms to smooth the output of the Condensation lter which improves the accuracy of motion estimation in a batch-mode procedure after tracking is complete.

This thesis addresses a number of issues concerned with the restoration of one type of image sequence, namely archived black and white motion pictures. These are often a valuable historical record, but because of the physical nature of the film they can suffer from a variety of degradations which reduce their usefulness. The main visual defects are `dirt and sparkle' due to dust and dirt becoming attached to the film, or abrasion removing the emulsion, and `line scratches' due to the film running against foreign bodies in the camera or projector. For an image

In this paper, we discuss a geometrical model of a space of deformable images or shapes, in which infinitesimal variations are combinations of elastic deformations (warping) and of photometric variations. Geodesics in this space are related to velocity-based image warping methods, which have proved to yield efficient and robust estimations of diffeomorphisms in the case of large deformation. Here, we provide a rigorous and general construction of this infinite dimensional “shape manifold ” on which we place a Riemannian metric. We then obtain the geodesic equations, for which we show the existence and uniqueness of solutions for all times. We finally use this to provide a geometrically founded linear approximation of the deformations of shapes in the neighborhood of a given template.

This paper addresses some problems in the interpretation of visually observed shapes in motion, both planar and three-dimensional shapes. Mumford (1996), interpreting the "Pattern Theory" developed over a number of years by Grenander (1976), views images as "pure" patterns that have been distorted by a combination of four kinds of degradations. This view applies naturally to the analysis of static, two-dimensional images. The four degradations are given here, together with comments on how they need to be extended to take account of three-dimensional objects in motion.

A common problem in Bayesian object recognition using marked point process models is to produce a point estimate of the true underlying object configuration. In the Bayesian framework we could use decision theory and the concept of loss functions to design a more reasonable estimator for the true underlying object configuration than the common zero--one loss, which corresponds to the maximum a--posteriori estimator. We propose to use the \Delta--metric of Baddeley (1992) as our loss function. It is demonstrated that the optimal Bayesian estimator corresponding to the \Delta--metric can be well approximated by combining Markov chain Monte Carlo methods with Simulated Annealing into a two-- step algorithm. The proposed loss function is tested using a marked point process model developed for locating cells in confocal microscopy images. In order to obtain reliable results, we must include moves that split and fuse objects within the Markov chain Monte Carlo framework. The exper...

The power of sampling methods in Bayesian reconstruction of noisy signals is well known. The extension of sampling to temporal problems is discussed. Efficacy of sampling over time is demonstrated with visual tracking.

this article is to discuss general reasons for this prominence of MCMC, to give an overview of a variety of image models and the use made of MCMC methods in dealing with them, to describe two applications in more detail, To appear as a chapter in the book Practical Markov chain Monte Carlo, edited by W. Gilks, S. Richardson and D. Spiegelhalter, published by Chapman and Hall.

In this paper, we discuss a geometrical modeling of a space of deformable shapes, in which innitesimal variations are combinations of elastic deformations (warping) and of photometric variations. Geodesics in this space are related to velocity-based image warping methods, which have proved to yield eOEcient and robust estimations of dioeeomorphisms in the case of large deformation. Here, we provide a rigourous and general construction of this innite dimensional ishape manifoldj, on which we place a Riemannian metric. We then obtain the geodesic equations, for which we show the existence and uniqueness of solutions for all times. We nally use this to provide a geometrically-founded linear approximation of the deformations of shapes in the neighbourhood of a given template. Contents 1.