In this report, I discuss the use of vision to support concrete, everyday activity. I will argue that a variety of interesting tasks can be solved using simple and inexpensive vision systems. I will provide a number of working examples in the form of a state-of-the-art mobile robot, Polly, which uses vision to give primitive tours of the seventh floor of the MIT AI Laboratory. By current standards, the robot has a broad behavioral repertoire and is both simple and inexpensive (the complete robot was built for less than $20,000 using commercial board-level components). The approach I will use will be to treat the structure of the agent's activity--- its task and environment---as positive resources for the vision system designer. By performing a careful analysis of task and environment, the designer can determine a broad space of mechanisms which can perform the desired activity. My principal thesis is that for a broad range of activities, the space of applicable mechanisms will be broad...

Abstract not found

We explore the use of the non-dominant hand to control a virtual camera while the dominant hand performs other tasks in a virtual 3D scene. Two experiments and an informal study are presented which evaluate this interaction style by comparing it to the status-quo unimanual interaction. In the first experiment, we find that for a target selection task, performance using the bimanual technique was 20% faster. Experiment 2 compared performance in a more complicated object docking task. Performance advantages are shown, however, only after practice. Free-form 3D painting was explored in the user study. In both experiments and in the user study participants strongly preferred the bimanual technique. The results also indicate that user preferences concerning bimanual interaction may be driven by factors other than simple time-motion performance advantages.

In natural vision, information overspecifies the relative distances between objects and their layout in three dimensions. Directed perception applies (Cutting, 1986), rather than direct or indirect perception, because any single source of information (or cue) might be adequate to reveal relative depth (or local depth order), but many are present and useful to observers. Such overspecification presents the theoretical problem of how perceivers use this multiplicity of information to arrive at a unitary appreciation of distance between objects in the environment. This article examines three models of directed perception: selection, in which only one source of information is used; addition, in which all sources are used in simple combination; and multiplication, in which interactions among sources can occur. To establish perceptual overspecification, we created stimuli with four possible sources of monocular spatial information, using all combinations of the presence or absence of relative size, height in the projection plane, occlusion, and motion parallax. Visual stimuli were computer generated and consisted of three untextured parallel planes arranged in depth. Three tasks were used: one of magnitude estimation of exocentric

This thesis presents a generic and principled solution for optimising the visual complexity of any arbitrary computer-generated virtual environment (VE). This is performed with the ultimate goal of reducing the inherent latencies of current virtual reality (VR) technology. Effectively, we wish to remove extraneous detail from an environment which the user cannot perceive, and thus modulate the graphical complexity of a VE with little or no perceptual artifacts. The work proceeds by investigating contemporary models and theories of visual perception and then applying these to the field of real-time computer graphics. Subsequently, a technique is devised to assess the perceptual content of a computer-generated image in terms of spatial frequency (c/deg), and a model of contrast sensitivity is formulated to describe a user's ability to perceive detail under various conditions in terms of this metric. This allows us to base the level of detail (LOD) of each object in a VE on a measure of ...

We show that the set of 2D images produced by the point features of a rigid 3D model can be represented with two lines in two high-dimensional spaces. These lines are the lowest-dimensional representation possible. We use this result to build a system for representing in a hash table at compile time, all the images that groups of model features can produce. Then at run time a group of image features can access the table and find all model groups that could match it. This table is efficient in terms of space, and is built and accessed through analytic methods that account for the effect of sensing error. In real images, it reduces the set of potential matches by a factor of several thousand. We also use this representation of a model's images to analyze two other approaches to recognition: invariants, and non-accidental properties. These are properties of images that some models always produce, and all other models either never produce (invariants) or almost never produce (non-accidental properties). In several domains we determine when invariants exist. In general we show that there are an infinite set of non-accidenta properties that are qualitatively similar.

Visual collision avoidance involves two difficult subproblems: obstacle recognition and depth measurement. We present a class of algorithms that use particularly simple methods for each subproblem and derive a set of sufficient conditions for their proper functioning based on a set of idealizations. We then discuss and compare two different implementations of the approach and discuss their performance. Finally, we experimentally validate the idealizations. 1 1 Introduction One of the major difficulties of visual obstacle avoidance is that obstacles can seemingly have arbitrary appearance. They can be tall or fat, plain or textured, rigid or flexible. Different algorithms can be seen as embodying different assumptions about the appearance of obstacles. Moravec's Stanford Cart [10] can be seen as assuming that obstacles (1) project significantly from the ground plane and (2) are densely covered with visual corners. Kriegman, Triendl and Binford's Mobi system [8] can be seen as assuming...

Visualizing information in user interfaces to complex, large-scale systems is difficult due to an enormous amount of dynamic data distributed across multiple displays. While graphical representation techniques can reduce some of the cognitive overhead associated with comprehension, current interfaces suffer from the over-use of such representation techniques and exceed the human’s perceptual capacity to efficiently interpret them. New display dimensions are required to support the user in information visualization. Three major issues which are problematic in complex system UI design are identified: representing the nature of change, supporting the cognitive integration of data across disparate displays, and conveying the nature of relationships between data and/ or events. Advances in technology have made animation a viable alternative to static representations. Motion holds promise as a perceptually rich and efficient display dimension but little is known about its attributes for information display. This paper proposes that motion may prove useful in visualizing complex information because of its preattentive and interpretative perceptual properties. A review of animation in current user interface and visualization design and research indicates that, while there is strong intuition about the “usefulness ” of motion to communicate, there

In this report we review a large body of literature describing how experience affects recognition. Both neurophysiology and psychophysics provide clear evidence for the development of recognition over time. In particular, we show how perceptual learning in recognition tasks can be directly linked to learning in feature tuned inferotemporal lobe neurons in the primate brain. The environment as we experience it, is so structured that potentially very different images appearing in close temporal succession are likely to be views of the same object. We argue that this temporal structure forms the basis of a tendency (a prior in the sense of Bayesian Statistics) of the human visual system to associate images of objects together over short periods of time.

Developers of autonomous capabilities underestimate the need for coordination with human team members when their automata are deployed into complex operational settings. Automata are brittle as literal minded agents and there is a basic asymmetry in coordinative competencies between people and automata. The new capabilities of robotic systems raise new questions about how to support coordination. This paper presents a series of issues that demand innovation to achieve human--robot coordination (HRC). These include supporting people in their roles as problem holder and as robotic handler, overcoming ambiguities in remote perception, avoiding coordination surprises by better tools to see into future robotic activities and contingencies, and responsibility in human--robot teams.