An object recognition system has been developed that uses a new class of local image features. The features are invariant to image scaling, translation, and rotation, and partially invariant to illumination changes and affine or 3D projection. These features share similar properties with neurons in inferior temporal cortex that are used for object recognition in primate vision. Features are efficiently detected through a staged filtering approach that identifies stable points in scale space. Image keys are created that allow for local geometric deformations by representing blurred image gradients in multiple orientation planes and at multiple scales. The keys are used as input to a nearest-neighbor indexing method that identifies candidate object matches. Final verification of each match is achieved by finding a low-residual least-squares solution for the unknown model parameters. Experimental results show that robust object recognition can be achieved in cluttered partially-occluded images with a computation time of under 2 seconds.

Model-based recognition and motion tracking depends upon the ability to solve for projection and model parameters that will best fit a 3-D model to matching 2-D image features. This paper extends current methods of parameter solving to handle objects with arbitrary curved surfaces and with any number of internal parameters representing articulations, variable dimensions, or surface deformations. Numerical

This paper provides a review of shape analysis methods. Shape analysis methods play an important role in systems for object recognition, matching, registration, and analysis. Researchin shape analysis has been motivated, in part, by studies of human visual form perception systems.

A robot acting in the real world must use flexible plans because actions will sometimes fail to produce desired effects, and unexpected events will sometimes demand the robot shift its attention. A plan is usually construed as a list of primitive robot actions to be executed one after another but in a complex domain, a plan must be structured to cope effectively with the myriad unpredictable details it will encounter during execution. However, adding structure to a plan involves more than augmenting the primitive plan representation; it requires a complete model of interaction with the world called situation-driven execution. Situation-driven execution assumes that a plan consists of tasks with three major components: a satisfaction test, a window of activity, and a set of execution methods that are appropriate in different circumstances. Execution of such a plan proceeds by selecting an unsatisfied t...

Research on human infants has begun to shed light on early-developing processes for segmenting perceptual arrays into objects. Infants appear to perceive obiects by analyzing three-dlmensional surface arrangements and motions. Their per-ception does not accord with a general tendency to maximize figural goodness or to attend to nonaccldentol geometric relations in visual arrays. Object perceptlan does accord with principles governing the motions of material bodies: Infants divide perceptual arrays into units that move as connected wholes, that move separately from one another, that tend to maintain their size ond shape over motion, and that tend to act upon each other only on contact. These findings sug-gest that a general representation of obiect unity and boundaries is interposed between representations of surfaces and representations of objects of famlllor kinds. The processes that construct this representotion may be related to pro-cesses of physical reasonlng. This article is animated by two proposals about perception and perceptual development. One proposal is substantive: In situations where perception develops through experience, but without instruction or deliberate reflection, development tends to enrich perceptual abilities but not to change them fundamentally. The second proposal is methodological: In the above situations, studies of the origins and early development of perception can shed light on perception in its mature state. These proposals will arise from a discussion of the early development of one perceptual ability: the ability to organize arrays of surfaces into unitary, bounded, and persisting objects. PERCEMNG OBJECTS In recent years, my colleagues and I have been studying young infants ’ perception of objects in complex displays in which objects are adjacent to other objects, objects are partly hidden behind other objects, or objects move fully Preparation of this article was supported by grants from NIH (I-ID-132r18) and NSF (BNS 06082). I am grateful to Carol Krumhansl, Doug Medin, and Herb Pick for penetrating com-ments on an earlier &aft of this manuscript. Correspondence and rquests for reprints should be sent to Elizabeth S. Spelke, Cornell

Abstract. Estimation techniques in computer vision applications must estimate accurate model parameters despite small-scale noise in the data, occasional large-scale measurement errors (outliers), and measurements from multiple populations in the same data set. Increasingly, robust estimation techniques, some borrowed from the statistics literature and others described in the computer vision literature, have been used in solving these parameter estimation problems. Ideally, these techniques should effectively ignore the outliers and measurements from other populations, treating them as outliers, when estimating the parameters of a single population. Two frequently used techniques are least-median of

How do we recognize objects despite differences in their retinal projections when they are seen at different orientations? Marr and Nishihara (1978) proposed that shapes are represented in memory as structural descriptions in objectcentered coordinate systems, so that an object is represented identically regardless of its orientation. An alternative hypothesis is that an object is represented in memory in a single representation corresponding to a canonical orientation, and a mental rotation operation transforms an input shape into that orientation before input and memory are compared. A third possibility is that shapes are stored in a set of representations, each corresponding to a different orientation. In four experiments, subjects studied several objects each at a single orientation, and were given extensive practice at naming them quickly, or at classifying them as normal or mirror-reversed, at several orientations. At first, response times increased with departure from the study orientation, with a slope similar to those obtained in classic mental rotation experiments. This suggests that subjects made both judgments by mentally transforming the orientation of the input shape to the one they

Abstract—We present a family of scale-invariant local shape features formed by chains of k connected roughly straight contour segments (kAS), and their use for object class detection. kAS are able to cleanly encode pure fragments of an object boundary without including nearby clutter. Moreover, they offer an attractive compromise between information content and repeatability and encompass a wide variety of local shape structures. We also define a translation and scale invariant descriptor encoding the geometric configuration of the segments within a kAS, making kAS easy to reuse in other frameworks, for example, as a replacement or addition to interest points (IPs). Software for detecting and describing kAS is released at

Accurate registration between real and virtual objects is crucial for augmented reality applications. Existing tracking methods are individually inadequate: magnetic trackers are inaccurate, mechanical trackers are cumbersome, and vision-based trackers are computationally problematic. We present a hybrid tracking method that combines the accuracy of vision-based tracking with the robustness of magnetic tracking without compromising real-time performance or usability. We demonstrate excellent registration in three sample applications.

There have been important recent advances in object recognition through the matching of invariant local image features. However, the existing approaches are based on matching to individual training images. This paper presents a method for combining multiple images of a 3D object into a single model representation. This provides for recognition of 3D objects from any viewpoint, the generalization of models to non-rigid changes, and improved robustness through the combination of features acquired under a range of imaging conditions. The decision of whether to cluster a training image into an existing view representation or to treat it as a new view is based on the geometric accuracy of the match to previous model views. A new probabilistic model is developed to reduce the false positive matches that would otherwise arise due to loosened geometric constraints on matching 3D and non-rigid models. A system has been developed based on these approaches that is able to robustly recognize 3D objects in cluttered natural images in sub-second times.