Diagrams often use repetition to convey points and establish contrasts. This paper shows how MAGI, our model of repetition and symmetry detection, can model the cognitive processes humans use when reading repetition-based diagrams. MAGI, which is based on the Structure Mapping Engine, detects repetition by aligning both visual and conceptual relational structure. This lets visual regularity of form support an understanding of the conceptual regularity such forms often depict. We describe JUXTA, which uses this insight to critique a class of diagrams that juxtapose similar scenes to demonstrate physical laws. Introduction In explanatory diagrams, repeated structures often have special significance. To underscore a point or emphasize a difference, diagrams often juxtapose events, scenes, or objects. Examples include a "before and after" display of shirts in a laundry detergent ad and a point-by-point comparison of pumps in a physics text. In such cases, visual repetition heightens cont...

Most existing sketch understanding systems require a closed domain to achieve recognition. This paper describes an incremental learning technique for opendomain recognition. Our system builds generalizations for categories of objects based upon previous sketches of those objects and uses those generalizations to classify new sketches. We represent sketches qualitatively because we believe qualitative information provides a level of description that abstracts away details that distract from classification, such as exact dimensions. Bayesian reasoning is used in building representations to deal with the inherent uncertainty in perception. Qualitative representations are compared using SME, a computational model of analogy and similarity that is supported by psychological evidence, including

This is Appendix A to the thesis " Learning to See Analogies: a Connectionist Exploration."

elland, J.L. (1983). Putting knowledge in its place: A scheme for programming parallel

We are trying to solve the problem of learning to recognize objects in an open-domain sketching environment. Our system builds generalizations of objects based upon previous sketches of those objects and uses those generalizations to classify new sketches. We represent sketches qualitatively because we believe qualitative information provides a level of description that abstracts away details that distract from classification, such as exact dimensions. Bayesian reasoning is used in the process of building up representations to deal with the inherent uncertainty in the perception problem. Qualitative representations are compared using SME, a computational model of analogy and similarity that is supported by psychological evidence from studies of perceptual similarity. We produce generalizations based on the common structure found by SME in different sketches of the same object. We report on the results of testing the system on a corpus of sketches of everyday objects, drawn by ten different people. 1.

An important problem in the integration of vision and language is comprehending explanatory diagrams, such as those found in science and engineering textbooks. One class of diagrams, which we call juxtaposition diagrams, illustrate a physical principle by comparing two similar situations that vary in a carefully chosen way. This paper describes research in progress on a computational model, JUXTA, which analyzes juxtaposition diagrams. JUXTA performs its analysis by finding the interesting differences in a figure, and then relating those differences to differences stated in the diagram caption. By using the visible differences in the figure as reference points for the qualitative relationship given in the caption, JUXTA is able to intelligently label the relevant parts of the figure. JUXTA also critiques the figure for understandability, warning of differences in the figure which may confuse the reader, and noting visible differences in the figure which are irrelevant and may be remove...

We propose that qualitative physics can provide an important component of natural language semantics. Specifically, we describe how qualitative process theory can be recast in terms of frame semantics, as used in the Berkeley FrameNet project.

Complex spatial and physical concepts are often communicated using diagrams. For many qualitative reasoning tasks, it is necessary that computers understand diagrams in much the same way as their human collaborators. Here we describe some preliminary work on basic diagram interpretation based on common depiction conventions. Using a combination of semantic and qualitative spatial information we are able to distinguish relevant regions and edges in sketched diagrams using the CogSketch sketch understanding system.

under Contract No. NIE-C-400-81-0030. It does not, however, necessarily