The perceptual recognition of objects is conceptualized to be a process in which the image of the input is segmented at regions of deep concavity into an arrangement of simple geometric components, such as blocks, cylinders, wedges, and cones. The fundamental assumption of the proposed theory, recognition-by-components (RBC), is that a modest set of generalized-cone components, called geons (N ^ 36), can be derived from contrasts of five readily detectable properties of edges in a two-dimensional image: curvature, collinearity, symmetry, parallelism, and cotermmation. The detection of these properties is generally invariant over viewing position and image quality and consequently allows robust object perception when the image is projected from a novel viewpoint or is degraded. RBC thus provides a principled account of the heretofore undecided relation between the classic principles of perceptual organization and pattern recognition: The constraints toward regularization (Pragnanz) characterize not the complete object but the object's components. Representational power derives from an allowance of free combinations of the geons. A Principle of Componential Recovery can account for the major phenomena of object recognition: If an arrangement of two or three geons can be recovered from the input, objects can be quickly recognized even when they are occluded, novel, rotated in depth, or extensively degraded. The results from experiments on the perception of briefly presented pictures by human observers provide empirical support for the theory. Any single object can project an infinity of image configura-tions to the retina. The orientation of the object to the viewer can vary continuously, each giving rise to a different two-dimen-sional projection. The object can be occluded by other objects or texture fields, as when viewed behind foliage. The object need not be presented as a full-colored textured image but in-stead can be a simplified line drawing. Moreover, the object can even be missing some of its parts or be a novel exemplar of its

How do we recognize objects despite differences in their retinal projections when they are seen at different orientations? Marr and Nishihara (1978) proposed that shapes are represented in memory as structural descriptions in objectcentered coordinate systems, so that an object is represented identically regardless of its orientation. An alternative hypothesis is that an object is represented in memory in a single representation corresponding to a canonical orientation, and a mental rotation operation transforms an input shape into that orientation before input and memory are compared. A third possibility is that shapes are stored in a set of representations, each corresponding to a different orientation. In four experiments, subjects studied several objects each at a single orientation, and were given extensive practice at naming them quickly, or at classifying them as normal or mirror-reversed, at several orientations. At first, response times increased with departure from the study orientation, with a slope similar to those obtained in classic mental rotation experiments. This suggests that subjects made both judgments by mentally transforming the orientation of the input shape to the one they

Is human object recognition viewpoint dependent or viewpointinvariant under #everyday " conditions? Biederman and Gerhardstein #1993# argue that viewpoint-invariant mechanisms are used almost exclusively.However, our analysis indicates that: 1# their conditions for immediate viewpointinvariance lack the generalitytocharacterize a wide range of recognition phenomena; 2# the extensive body of viewpoint-dependent results cannot be dismissed as processing #by-products" or #experimental artifacts"; 3# geon structural descriptions cannot coherently account for category recognition, the domain they are intended to explain. We conclude that the weight of current evidence supports an exemplar-based multiple-views mechanism as an important component of both exemplar-speci#c and categorical recognition. # Many of the ideas in this paper were developed during visits by MJT to the Max#Planck#Institut f#ur biologische Kybernetik in T#ubingen, Germany.We thank Dan Kersten for his insightful comments...

This paper is a study of techniques for measuring and predicting visual fidelity. As visual stimuli we use polygonal models, and vary their fidelity with two different model simplification algorithms. We also group the stimuli into two object types: animals and man made artifacts. We examine three different experimental techniques for measuring these fidelity changes: naming times, ratings, and preferences. All the measures were sensitive to the type of simplification and level of simplification. However, the measures differed from one another in their response to object type. We also examine several automatic techniques for predicting these experimental measures, including techniques based on images and on the models themselves. Automatic measures of fidelity were successful at predicting experimental ratings, less successful at predicting preferences, and largely failures at predicting naming times. We conclude with suggestions for use and improvement of the experimental and automatic measures of visual fidelity.

Theories of visual object recognition must solve the problem of recognizing 3D objects given that perceivers only receive 2D patterns of light on their retinae. Recent findings from human psychophysics, neurophysiology and machine vision provide converging evidence for `image-based' models in which objects are represented as collections of viewpoint-specific local features. This approach is contrasted with `structural-description' models in which objects are represented as configurations of 3D volumes or parts. We then review recent behavioral results that address the biological plausibility of both approaches, as well as some of their computational advantages and limitations. We conclude that, although the image-based approach holds great promise, it has potential pitfalls that may be best overcome by including structural information. Thus, the most viable model of object recognition may be one that incorporates the most appealing aspects of both image-based and structural-description theories. 1998 Elsevier Science B.V. All rights reserved Keywords: Object recognition; Image-based model; Structural description 1.

this paper, the results of these neurophysiological studies, and the results of previous behavioral studies offer persuasive evidence that object recognition is an inherently viewpointdependent process. This perspective is embodied in a host of theories [13, 14, 15, 16] which assume that collections of features, surfaces, parts, or entire images of objects are represented in a viewpoint-specific manner. According to these theories, recognition of test objects is based on the similarity between studied images and tested images. Objects seen from viewpoints increasingly different from learned views will project increasingly less-similar images, so view-based theories provide a natural account for viewpoint effects found here and in other recent recognition studies [7, 17].

A scheme for recognizing 3D objects from single 2D images is introduced. The scheme proceeds in two stages. In the first stage, the categorization stage, the image is compared to prototype objects. For each prototype, the view that most resembles the image is recovered, and, if the view is found to be similar to the image, the class identity of the object is determined. In the second stage, the identification stage, the observed object is compared to the individual models of its class, where classes are expected to contain objects with relatively similar shapes. For each model, a view that matches the image is sought.

The use of outline shape in recognizing objects was investigated in four experiments. In Experiment 1, subjects matched a shaded image to either another shaded image or a silhouette. In Experiment 2, subjects initially named shaded images; later they named either shaded images or silhouettes. Performance in both experiments was predicted by changes in the outline shape of the stimuli. The same matching (Experiment 3) and priming (Experiment 4) paradigms were then used to investigate recognition with objects that were rotated between presentations so as to change the outline shape of the object. Recognition was predicted by changes to outline shape. These results place constraints on models of object recognition, and are most compatible with viewpoint-dependent models of recognition.

this paper is to investigate the nature of image-based class generalization,

this paper appear incompatible with Biederman's (1987) RBC model of recognition and, specifically, many of the studies supporting the RBC model that seem to show immediate viewpoint invariance (e.g., Biederman & Gerhardstein, 1993). In particular, our Experiment 1 employed a methodology quite similar to that of Biederman and Gerhardstein's (1993) Experiment 3, but showed systematic viewpoint dependence even though changes in viewpoint did not produce a change in the visible parts. One reason for this difference may be that in Biederman and Gerhardstein's experiment, changes in viewpoint that retained the same visible parts resulted in roughly mirror-reflected image pairs. In our experiment rotations in depth never crossed the bilaterallysymmetric front of the object, and so changes in viewpoint never resulted in two images of the object that were mirror-reflections. This difference may be crucial, as there is evidence that mirror-image transformations can be achieved with little or no cost to recognition performance (e.g. Biederman & Cooper, 1991; Cooper et al., 1992). Recognition of unfamiliar views may, therefore, be particularly efficient across mirror-image transformations.