This paper presents a method for extracting distinctive invariant features from images, which can be used to perform reliable matching between different images of an object or scene. The features are invariant to image scale and rotation, and are shown to provide robust matching across a a substantial range of affine distortion, addition of noise, change in 3D viewpoint, and change in illumination. The features are highly distinctive, in the sense that a single feature can be correctly matched with high probability against a large database of features from many images. This paper also describes an approach to using these features for object recognition. The recognition proceeds by matching individual features to a database of features from known objects using a fast nearest-neighbor algorithm, followed by a Hough transform to identify clusters belonging to a single object, and finally performing verification through leastsquares solution for consistent pose parameters. This approach to recognition can robustly identify objects among clutter and occlusion while achieving near real-time performance.

Abstract. In this paper, we present a novel scale- and rotation-invariant interest point detector and descriptor, coined SURF (Speeded Up Robust Features). It approximates or even outperforms previously proposed schemes with respect to repeatability, distinctiveness, and robustness, yet can be computed and compared much faster. This is achieved by relying on integral images for image convolutions; by building on the strengths of the leading existing detectors and descriptors (in casu, using a Hessian matrix-based measure for the detector, and a distribution-based descriptor); and by simplifying these methods to the essential. This leads to a combination of novel detection, description, and matching steps. The paper presents experimental results on a standard evaluation set, as well as on imagery obtained in the context of a real-life object recognition application. Both show SURF’s strong performance. 1

Abstract. Methods based on local, viewpoint invariant features have proven capable of recognizing objects in spite of viewpoint changes, occlusion and clutter. However, these approaches fail when these factors are too strong, due to the limited repeatability and discriminative power of the features. As additional shortcomings, the objects need to be rigid and only their approximate location is found. We present an object recognition approach which overcomes these limitations. An initial set of feature correspondences is first generated. The method anchors on it and then gradually explores the surrounding area, trying to construct more and more matching features, increasingly farther from the initial ones. The resulting process covers the object with matches, and simultaneously separates the correct matches from the wrong ones. Hence, recognition and segmentation are achieved at the same time. Only very few correct initial matches suffice for reliable recognition. Experimental results on still images and television news broadcasts demonstrate the stronger power of the presented method in dealing with extensive clutter, dominant occlusion, large scale and viewpoint changes. Moreover non-rigid deformations are explicitly taken into account, and the approximative contours of the object are produced. The approach can extend any viewpoint invariant feature extractor. 1

In this paper we describe a novel technique for detecting salient regions in an image. The detector is a generalization to a#ne invariance of the method introduced by Kadir and Brady [10]. The detector deems a region salient if it exhibits unpredictability in both its attributes and its spatial scale.

This article presents a novel scale- and rotation-invariant detector and descriptor, coined SURF (Speeded-Up Robust Features). SURF approximates or even outperforms previously proposed schemes with respect to repeatability, distinctiveness, and robustness, yet can be computed and compared much faster. This is achieved by relying on integral images for image convolutions; by building on the strengths of the leading existing detectors and descriptors (specifically, using a Hessian matrix-based measure for the detector, and a distribution-based descriptor); and by simplifying these methods to the essential. This leads to a combination of novel detection, description, and matching steps. The paper encompasses a detailed description of the detector and descriptor and then explores the effect of the most important parameters. We conclude the article with SURF’s application to two challenging, yet converse goals: camera calibration as a special case of image registration, and object recognition. Our experiments underline SURF’s usefulness in a broad range of topics in computer vision.

We present a novel approach for establishing multiple-view feature correspondences along an unordered set of images taken from substantially different viewpoints. While recently several wide-baseline stereo (WBS) algorithms have appeared, the N-view case is largely unexplored. In this paper, an established WBS algorithm is used to extract and match features in pairs of views. The pairwise matches are first integrated into disjoint feature tracks, each representing a single physical surface patch in several views. By exploiting the interplay between the tracks, they are extended over more views, while unrelated image features are removed. Similarity and spatial relationships between the features are simultaneously used. The output consists of many reliable and accurate feature tracks, strongly connecting the input views. Applications include 3D reconstruction and object recognition. The proposed approach is not restricted to the particular choice of features and matching criteria. It can extend any method that provides feature correspondences between pairs of images. 1.

We present a novel approach for tracking locally planar regions in an image sequence and their grouping into larger planar surfaces. The tracker recovers the affine transformation of the region and therefore yields reliable point correspondences between frames. Both edges and texture information are exploited in an integrated way, while not requiring the complete region's contour. The tracker withstands zoom, out-of-plane rotations, discontinuous motion and changes in illumination conditions while achieving real-time performance for a region. Multiple tracked regions are grouped into disjoint coplanarity classes. We first define a coplanarity score between each pair of regions, based on motion and texture cues. The scores are then analyzed by a clique-partitioning algorithm yielding the coplanarity classes that best fit the data. The method works in the presence of perspective distortions, discontinuous planar surfaces and considerable amounts of measurement noise.

We present a new method for matching line segments between two uncalibrated wide-baseline images. Most current techniques for wide-baseline matching are based on viewpoint invariant regions. Those methods work well with highly textured scenes, but fail with poorly textured ones. We show that such scenes can be successfully matched using line segments. Moreover, since line segments and regions provide complementary information, their combined matching allows to deal with a broader range of scenes. We generate an initial set of line segment correspondences, and then iteratively increase their number by adding matches consistent with the topological structure of the current ones. Finally, a coplanar grouping stage allows to estimate the fundamental matrix even from line segments only.

A new and fast way to find local image correspondences for wide baseline image matching is described. The targeted application is visual navigation, e.g. of a semi-automatic wheelchair. Such applications pose some additional requirements, like the need to work with natural landmarks rather than artificial markers, and the need to recognize locations fast. The restricted motion of the camera can be exploited to simplify the feature extraction. These features should support their identification from different, but nevertheless restricted viewing directions, and under variable illumination conditions. The paper proposes a specialization of so-called affine invariant regions for these particular conditions, which in this case simplifies to column segments. Their applicability is wider than robot navigation, and includes localization for wearable computing and scene recognition for automatic movie indexing. 1

We present a novel Object Recognition approach based on affine invariant regions. It actively counters the problems related to the limited repeatability of the region detectors, and the difficulty of matching, in the presence of large amounts of background clutter and particularly challenging viewing conditions. After producing an initial set of matches, the method gradually explores the surrounding image areas, recursively constructing more and more matching regions, increasingly farther from the initial ones. This process covers the object with matches, and simultaneously separates the correct matches from the wrong ones. Hence, recognition and segmentation are achieved at the same time. The approach includes a mechanism for capturing the relationships between multiple model views and exploiting these for integrating the contributions of the views at recognition time. This is based on an efficient algorithm for partitioning a set of region matches into groups lying on smooth surfaces. Integration is achieved by measuring the consistency of configurations of groups arising from different model views. Experimental results demonstrate the stronger power of the approach in dealing with extensive clutter, dominant occlusion, and large scale and viewpoint changes. Non-rigid deformations are explicitly taken into account, and the approximative contours of the object are produced. All presented techniques can extend any viewpoint invariant feature extractor. 1