We report results from perceptual judgment, delayed matching to sample, and long-term memory recall experiments, which indicate that the human visual system can support metrically veridical representations of similarities among 3D objects. In all the experiments, animal-like computer-rendered stimuli formed regular planar configurations in a common 70-dimensional parameter space. These configurations were fully recovered by multidimensional scaling from proximity tables derived from the subject data. We show that such faithful representation of similarity is possible if shapes are encoded by their similarities to a number of reference (prototypical) shapes, as in the computational model that accompanies the psychophysical data. Keywords: Object recognition / Representation / Similarity / Multidimensional scaling Running title: Faithful Representation of Similarity To whom correspondence should be addressed. 1 Introduction The human visual system possesses an impressive ability to...

An important element of learning from examples is the extraction of patterns and regularities from data. This paper investigates the structure of patterns in data defined over discrete features, i.e. features with two or more qualitatively distinct values. Any such pattern can be algebraically decomposed into a spectrum of component patterns, each of which is a simpler or more atomic ‘‘regularity.’ ’ Each component regularity involves a certain number of features, referred to as its degree. Regularities of lower degree represent simpler or more coarse patterns in the original pattern, while regularities of higher degree represent finer or more idiosyncratic patterns. The full spectral breakdown of a pattern into component regularities of minimal degree, referred to as its power series, expresses the original pattern in terms of the regular rules or patterns it obeys, amounting to a kind of ‘‘theory’ ’ of the pattern. The number of regularities at various degrees necessary to represent the pattern is tabulated in its power spectrum, which expresses how much of a pattern’s structure can be explained by regularities of various levels of complexity. A weighted mean of the pattern’s spectral power gives a useful numeric summary of its overall complexity, called its algebraic complexity. The basic theory of algebraic decomposition is extended in several ways, including algebraic accounts of the typicality of individual objects within concepts, and estimation of the power series from noisy data. Finally some relations between these algebraic quantities and empirical data are discussed.

We show that a certain class of similarity measures, which is based on set-theoretic concepts, and explains many of the characteristics of human similarity assessment, can be interpreted as a distance in a suitable psychological space. This view unifies a number of di#erent measures of similarity that psychological experiments have determined to be active in humans for di#erent classes of stimuli. 1 Introduction This paper speculates on some of the ideas we are working on while trying to build a multimedia database based on similarity. The problem is of very wide scope, and takes together issues from the most diverse fields: we have system problems for the e#- cient organization of the huge mass of data, network problems for their distribution over a wide geographic area, query optimization problems, and on and on and on. The core of the whole business, though, will be the similarity assessment operation. At a certain point in the database, two images must be compared and the similar...

One of the central problems in cognitive science is determining the mental representations that underlie human inferences. Solutions to this problem often rely on the analysis of subjective similarity judgments, on the assumption that recognizing “likenesses ” between people, objects and events is crucial to everyday inference. One such solution is provide by the additive clustering model, which is widely used to infer the features of a set of stimuli from their similarities, on the assumption that similarity is a weighted linear function of common features. Existing approaches for implementing additive clustering often lack a complete framework for statistical inference, particularly with respect to choosing the number of features. To address these problems, this paper develops a fully Bayesian formulation of the additive clustering model, using methods from nonparametric Bayesian statistics to allow the number of features to vary. We use this to explore several approaches to parameter estimation, showing that the nonparametric Bayesian approach provides a straightforward way to obtain estimates of both the number of features and their importance. 1

This paper presents some methodological observations on the measurement of performance in Visual Information Retrieval (VIR) systems. The paper identifies three di#erent types of measures two of which (Physical Performance and Contextual Evaluation) can be determined with methods inherited from physical and social sciences respectively. The third model (Decontextualized Evaluation) is more typical of the design and construction of complicated systems, since it allows us to measure the performance of individual modules before their insertion in a particular application. This paper presents some methodologies for the decontextualized evaluation, anchoring them to a case studies of evaluation of several subsystems of an image database. Keywords: Visual Information Systems, Evaluation 1. INTRODUCTION In this paper I present some methodological observations about the problem of evaluating Visual Information Retrieval (VIR) systems. My observations, and the examples that I will present, a...

It is well known that pairs of dimensions that are processed holistically- integral dimensions- normally combine with a Euclidean metric, whereas pairs of dimensions that are processed analytically- separable dimensions- most often combine with a city-block metric. This paper extends earlier research regarding information integration in that it deals with complex stimuli consisting of both dimensional pairs previously identified as holistic, and dimensional pairs previously identified as analytical. The general pattern identified is that information integration can be more accurately described with a rule taking aspects of stimuli into consideration compared to a traditional rule. For example, it appears that combinations of analytical and holistic stimuli, are better described by treating the different subspaces individually and then combining these with addition, compared to any single Minkowskian rule, and much better compared to any of the Minkowskian rules traditionally used (i.e. the cityblock-, the Euclidean or the dominance-metrics).

Pen-based user interfaces are becoming ever more popular. Gestures (i.e., marks made with a pen to invoke a command) are a valuable aspect of pen-based UIs, but they also have drawbacks. The challenge in designing good gestures is to make them easy for people to learn and remember. With the goal of better gesture design, we performed a pair of experiments to determine why users find gestures similar. From these experiments, we have derived a computational model for predicting perceived gesture similarity that correlates 0.56 with observation. We will incorporate the results of these experiments into a gesture design tool, which will aid the pen-based UI designer in creating gesture sets that are easier to learn and more memorable. Keywords Pen-based user interfaces, pen gestures, multi-dimensional scaling, similarity, perception

Perception of facial expressions of emotion is generally assumed to correspond to underlying muscle movement. However, it is often observed that some individuals have sadder or angrier faces, even for neutral, motionless faces. Here, we report on one such effect caused by simple static configural changes. In particular, we show four variations in the relative vertical position of the nose, mouth, eyes, and eyebrows that affect the perception of emotion in neutral faces. The first two configurations make the vertical distance between the eyes and mouth shorter than average, resulting in the perception of an angrier face. The other two configurations make this distance larger than average, resulting in the perception of sadness. These perceptions increase with the amount of configural change, suggesting a representation based on variations from a norm (prototypical) face.

With complex multimedia data, we see the emergence of database systems in which the fundamental operation is similarity assessment. Before database issues can be addressed, it is necessary to give a definition of similarity as an operation. In this paper we study several models proposed in the psychological literature to explain the characteristics of human similarity assessment, and discuss their computational characteristics. In addition, we develop a similarity measure, based on fuzzy logic, that exhibit several features that match experimental findings in humans. The model is dubbed Fuzzy Feature Contrast (FFC) and is an extension to a more general domain of the Feature Contrast model due to Tversky. We show how the FFC model can be used to model similarity assessment from fuzzy judgment of properties, and we address the use of fuzzy measures to deal with dependencies among the properties. Finally, we show how the parameters of the model can be tuned by comparative judgment experim...

v Declaration .................................... ix Acknowledgements................................ xi 1Prelude 1 TheVeryIdeaofRepresentation......................... 2 TypesofSimilarity ................................ 8 IsSimilarityIndeterminate? ........................... 11 TheRoleofSimilarityinCognition....................... 11 Summary&GeneralDiscussion......................... 14 2 Theories of Similarity 17 SimilarityDataSets................................ 17 SpatialRepresentation .............................. 21 FeaturalRepresentation.............................. 31 TreeRepresentation................................ 40 NetworkRepresentation ............................. 47 Alignment-BasedSimilarityModels....................... 48 TransformationalSimilarityModels ....................... 50 Summary&GeneralDiscussion......................... 54 i 3 On Representational Complexity 55 ApproachestoModelSelection ......................... 57 ChoosinganAdditiveClusteringRepresentation ................ 67 ChoosinganAdditiveTreeRepresentation ................... 82 ChoosingaSpatialRepresentation........................ 94 Summary&GeneralDiscussion......................... 95 4 Featural Representation 97 AMenagerieofFeaturalModels......................... 98 ClusteringModels.................................104 GeometricComplexityCriteria..........................106 AlgorithmsforFittingFeaturalModels .....................107 MonteCarloStudyI:DotheAlgorithmsWork? ................109 RepresentationsofKinshipTerms ........................117 MonteCarloStudyII:Complexity........................122 ExperimentI:Faces................................125 ExperimentII:Countries .............................1...