This paper describes how agents can learn an internal model of the world structurally by focusing on the problem of behavior-based articulation. We develop an on-line learning scheme -- the so-called mixture of recurrent neural net (RNN) experts -- in which a set of RNN modules becomes self-organized as experts on multiple levels in order to account for the different categories of sensory-motor flow which the robot experiences. Autonomous switching of activated modules in the lower level actually represents the articulation of the sensory-motor flow. In the meanwhile, a set of RNNs in the higher level competes to learn the sequences of module switching in the lower level, by which articulation at a further more abstract level can be achieved. The proposed scheme was examined through simulation experiments involving the navigation learning problem. Our dynamical systems analysis clarified the mechanism of the articulation; the possible correspondence between the articulation...

We propose that the ability to extract regularities from time series through prediction learning can be enhanced if we use a hierarchical architecture in which higher layers are trained to predict the internal state of lower layers when such states change significantly. This hierarchical organization has two functions: (a) it forces the system to progressively re-code sensory information so as to enhance useful regularities and filter out useless information; (b) it progressively reduces the length of the sequences which should be predicted going from lower to higher layers. This, in turn, allows higher levels to extract higher level regularities which are hidden at the sensory level. By training an architecture of this type to predict the next sensory state of a robot navigating in a environment divided into two rooms we show how the first level prediction layer extracts low level regularities such as `walls', `corners', and `corridors' while the second level prediction laye...

The explicit investigation of anticipations in relation to adaptive behavior is a recent approach. This chapter first provides psychological background that motivates and inspires the study of anticipations in the adaptive behavior field. Next, a basic framework for the study of anticipations in adaptive behavior is suggested. Different anticipatory mechanisms are identified and characterized. First fundamental distinctions are drawn between implicit anticipatory behavior, payoff anticipatory behavior, sensory anticipatory behavior, and state anticipatory behavior. A case study allows further insights into the drawn distinctions.

This study attempts to describe the notion of the "self " using dynamical systems language based on the results of our robot learning experiments. A neural network model consisting of multiple modules is proposed, in which the interactive dynamics between the bottom-up perception and the top-down prediction are investigated. Our experiments with a real mobile robot showed that the incremental learning of the robot switches spontaneously between steady and unsteady phases. In the steady phase, the top-down prediction for the bottom-up perception works well when coherence is achieved between the internal and the environmental dynamics. In the unsteady phase, con icts arise between the bottom-up perception and the top-down prediction; the coherence is lost, and a chaotic attractor is observed in the internal neural dynamics. By investigating possible analogies between this result and the phenomenological literature on the "self", we draw the conclusions that (1) the structure of the "self" corresponds to the "open dynamic structure " which ischaracterized by co-existence of stability in terms of goal-directedness and instability caused by embodiment; (2) the open dynamic structure causes the system's spontaneous transition to the unsteady phase where the "self " becomes aware. 1

Autonomous agents make frequent use of knowledge in the form of categories --- categories of objects, human gestures, web pages, and so on. This paper describes a way for agents to learn such categories for themselves through interaction with the environment. In particular, the learning algorithm transforms raw sensor readings into clusters of time series that have predictive value to the agent. We address several issues related to the use of an uninterpreted sensory apparatus and show specific examples where a Pioneer 1 mobile robot interacts with objects in a cluttered laboratory setting.

Abstract not found

, and inferential coherence. They lack these aspects because their underlying methods have been unable to deal eectively with constituent structure, though more elaborate implementations should overcome this limitation (see, for example, Chalmers 1993). We tend to think of these aspects of thinking as being typically human. This is probably because they were rst studied in relation to language (Katz &Fodor 1963; Chomsky 1959,1965). Human thinking is systematic insofar as people who can understand one sentence (e.g., 869F<uIU 1998 G/9 7n31 F| %-!<%o!<%I!' Aordances, Cognitive Robotics, Consciousness, Frame Problem, Machine Learning, Symbol Grounding 3 Kisokogakubu, Toyonaka, Osaka ")560-8531 John loves Mary) can, in general, understand structurally similar sentences (Mary loves John); it is productive insofar as people can understand and generate an unbounded number of sentences; and i

this paper, a network of the former type will be analyzed in the following. Figure 24 shows a characteristic trajectory of a successful robot controller of this type. As above, the robot starts off facing the upper left obstacle. It turns away from it to the left, enters the zone, and collects three objects on its first pass through the zone, turning slightly to the left towards each of them. As soon as it has left the zone it starts moving in a semi-circle to the left, which takes it back into the zone. In the zone it starts moving straight ahead again, takes a slight turn to the right to collect the upper object, and continues straight ahead out of the zone. The same pattern is repeated: as soon as it leaves the zone, it moves in a semi-circle to the left, which takes it back into the zone, where it starts moving straight forward again. Once more it performs a slight turn to the right to collect an object it would otherwise have missed. It continues to move straight ahead, leaves the zone, returns in another semi-circle, enters once more and moves straight ahead until the evaluation period ends.

Two classes of dynamical recurrent neural

Abstract. This paper explores the capabilities of continuous time recurrent neural networks (CTRNNs) to display reinforcement learning-like abilities on a set of T-Maze and double T-Maze navigation tasks, where the robot has to locate and “remember ” the position of a reward-zone. The “learning ” comes about without modifications of synapse strengths, but simply from internal network dynamics, as proposed by [12]. Neural controllers are evolved in simulation and in the simple case evaluated on a real robot. The evolved controllers are analyzed and the results obtained are discussed. 1