Constraints and dependencies among the elements of embodied cognition form patterns or microstrategies of interactive behavior. Hard constraints determine which microstrategies are possible. Soft constraints determine which of the possible microstrategies are most likely to be selected. When selection is non-deliberate or automatic the least effort microstrategy is chosen. In calculating the effort required to execute a microstrategy each of the three types of operations, memory retrieval, perception, and action, are given equal weight; that is, perceptual-motor activity does not have a privileged status with respect to memory. Soft constraints can work contrary to the designer's intentions by making the access of perfect knowledge in-the-world more effortful than the access of imperfect knowledge in-the-head. These implications of soft constraints are tested in two experiments. In experiment 1 we varied the perceptual-motor effort of accessing knowledge in-the-world as well as the effort of retrieving items from memory. In experiment 2 we replicated one of the experiment 1 conditions to collect eye movement data. The results suggest that milliseconds matter. Soft constraints lead to a reliance on knowledge in-the-head even when the absolute difference in perceptual-motor versus memory retrieval effort is small, and even when relying on memory leads to a higher error rate and lower performance. We discuss the implications of # Supplementary data associated with this article can be found, in the online version, at doi:10.1016/ j.cogsci.2003.12.001 ## An earlier, much simpler version of this report was presented as an eight-page conference paper at CHI2001. That paper is archived as Gray and Fu (2001).

This paper describes experiments performed with forty subjects wearing an eye-tracker and watching and imitating videos of finger, hand, and arm movements. For all types of stimuli, the subjects tended to fixate on the hand, regardless of whether they were imitating or just watching. The results lend insight into the connection between visual perception and motor control, suggesting that: 1) people analyze human arm movements largely by tracking the hand or the end-point, even if the movement is performed with the entire arm, and 2) when imitating, people use internal innate and learned models of movement, possibly in the form of motor primitives, to recreate the details of whole-arm posture and movement from end-point trajectories. Keywords: Perceptual-motor interaction; Eye-tracking; Movement imitation Theme: Motor Systems and Sensorimotor Integration Topic: Control of Posture and Movement 1 Introduction Imitation is one of the most ubiquitous forms of human learning. What appea...

To operate e#ectively in complex environments learning agents have to selectively ignore irrelevant details by forming useful abstractions. In this article we outline a formulation of abstraction for reinforcement learning approaches to stochastic sequential decision problems modeled as semiMarkov Decision Processes (SMDPs). Building on existing algebraic approaches, we propose the concept of SMDP homomorphism and argue that it provides a useful tool for a rigorous study of abstraction for SMDPs. We apply this framework to di#erent classes of abstractions that arise in hierarchical systems and discuss relativized options, a framework for compactly specifying a related family of temporally-extended actions. Additional details of this work are described in refs. [1, 2, 3].

In this paper, we rst discuss the meaning of physical embodiment and the complexity of the environment in the context of multiagent learning. We then propose a vision-based reinforcement learning method that acquires cooperative behaviors in a dynamic environment. We use the robot soccer game initiated by RoboCup [12] to illustrate the eectiveness of our method. Each agent works with other team members to achieve a common goal against opponents. Our method estimates the relationships between a learner's behaviors and those of other agents in the environment through interactions (observations and actions) using a technique from system identication. In order to identify the model of each agent, Akaike's Information Criterion is applied to the results of Canonical Variate Analysis to clarify the relationship between the observed data in terms of actions and future observations. Next, reinforcement learning based on the estimated state vectors is performed to obtain the optimal behavior...

This article presents an implemented multi-robot system for playing the popular game of laser tag. The object of the game is to search for and tag opponents that can move freely about the environment. The main contribution of this paper is a new variable-dimension particle filter algorithm for tracking the location of opponents under prolonged periods of occlusion. This algorithm can cope efficiently with variable numbers of opponents, through mechanisms that dynamically increase and decrease the number of particle tracks. When searching for opponents, the individual agents greedily maximize their information gain, using a negotiation technique for coordinating their search efforts. Experimental results are provided, obtained with a physical robot system in large-scale indoor environments.

Soft constraints hypothesis (SCH) is a rational analysis approach that holds that the mixture of perceptual-motor and cognitive resources allocated for interactive behavior is adjusted based on temporal cost-benefit tradeoffs. Alternative approaches maintain that cognitive resources are in some sense protected or conserved in that greater amounts of perceptual-motor effort will be expended to conserve lesser amounts of cognitive effort. One alternative, the minimum memory hypothesis (MMH), holds that people favor strategies that minimize the use of memory. SCH is compared with MMH across 3 experiments and with predictions of an Ideal Performer Model that uses ACT-R’s memory system in a reinforcement learning approach that maximizes expected utility by minimizing time. Model and data support the SCH view of resource allocation; at the under 1000-ms level of analysis, mixtures of cognitive and perceptual-motor resources are adjusted based on their cost-benefit tradeoffs for interactive behavior.

Visual cognition depends critically on the ability to make rapid eye movements known as saccades that orient the fovea over targets of interest in a visual scene. Saccades are known to be ballistic: the pattern of muscle activation for foveating a prespecified target location is computed prior to the movement and visual feedback is precluded. Despite these distinctive properties, there has been no general model of the saccadic targeting strategy employed by the human visual system during visual search in natural scenes. This paper proposes a model for saccadic targeting that uses iconic scene representations derived from oriented spatial filters at multiple scales. Visual search proceeds in a coarse-to-fine fashion with the largest scale filter responses being compared first. The model was empirically tested by comparing its performance with actual eye movement data from human subjects in a natural visual search task; preliminary results indicate substantial agreement between eye movements predicted by the model and those recorded from human subjects.

... In this article I define a taxonomy and three dimensions of simulated task environments. The dimensions are based on viewing simulated task environments from the perspectives of the researcher, the task, and the participants. Research on complex systems is inherently complex. It is my hope that the terms and distinctions introduced in this article will further the scientific enterprise by enabling us to spend less time explaining our paradigms and more time communicating our results

This paper presents the case for a functional account of vision. A variety of studies have consistently revealed “change blindness ” or insensitivity to changes in the visual scene during an eye movement. These studies indicate that only a small part of the information in the scene is represented in the brain from moment to moment. It is still unclear, however, exactlywhat is included in visual representations. This paper reviews experiments using an extended visuo-motor task, showing that display changes affect performance differently depending on the observer’s place in the task. These effects are revealed by increases in fixation duration following a change. Different task-dependent increases suggest that the visual system represents only the information that is necessary for the immediate visual task. This allows a principled exploration of the stimulus properties that are included in the internal visual representation. The task specificity also has a more general implication that vision should be conceptualized as an active process executing special purpose “routines ” that compute only the currently necessary information. Evidence for this view and its implications for visual representations are discussed. Comparison of the change blindness phenomenon and fixation durations shows that conscious report does not reveal the extent of the representations computed by the routines.

Abstract. The embodiment hypothesis is the idea that intelligence emerges in the interaction of an agent with an environment and as a result of sensorimotor activity. In this paper we offer six lessons for developing embodied intelligent agents suggested by research in developmental psychology. We argue that starting as a baby grounded in a physical, social and linguistic world is crucial to the development of the flexible and inventive intelligence that characterizes humankind.