We propose a research strategy to study the evolution of affective states and analyze the requirements for simulated environments to be appropriate for experiments with affective agent architectures. We present the simulation model and agent architecture used in our experiments to demonstrate that (1) primitive emotional states (such as "fear" and "anger") and primitive motivational states (such as "hunger" and "thirst") can play an important role in the control and coordination of agents in agent societies, and (2) such states are very likely to evolve (in certain environments).

Drawing from ideas in both traditional animation and modern philosophy, we present a methodology for designing synthetic characters. The goal of our approach is to construct intentional characters that are both compelling, in the sense that people can empathize with them, and understandable, in that their actions can be seen as attempts to satisfy their desires given their beliefs. We also present a simple, value-based framework that has the flexibility to implement the subsystems necessary for the construction of intentional characters.

Abstract. For two years, we have been involved in a challenging project to develop a new architecture for an industrial transportation system. The motivating quality attributes to develop this innovative architecture were flexibility and openness. Taking these quality attributes into account, we proposed a decentralized architecture using multiagent systems (MASs). A MAS consists of multiple autonomous entities that coordinate with each other to achieve decentralized control. The typical advantages attributed to such decentralized architecture are flexibility and openness, the motivating quality attributes to apply MAS in this case. The Architecture Tradeoff Analysis Method (ATAM) was used to provide insights wether our architecture meets the expected flexibility and openness, and to identify tradeoffs with other quality attributes. Applying the ATAM proved to be a valuable experience. One of the main outcome of applying the ATAM was the identification of a tradeoff between flexibility and communication load that results from the use of a decentralized architecture. This paper describes our experiences in applying the ATAM to a MAS architecture, containing both the main outcomes of the evaluation and a critical reflection on the ATAM itself. 1

The goal of creating an integrated cognitive robot is still only a tantalizing dream. Current artificial intelligence and robotics research is highly divergent with little or no commonality among specialized subfields. New rich task domains are needed to pose the right challenges to extant theories and promote convergence. We propose soccerplaying as such a task since it requires situated robotics, perception, real-time decision making, planning, plan recognition, learning and multirobot coordination and control. The technology to perform real-time vision and build autonomous robots is available; the Dynamite testbed has been built to perform experiments with multiple robots.

. The behaviour of reinforcement learning (RL) algorithms is best understood in completely observable, finite state- and action-space, discrete-time controlled Markov-chains. Robot-learning domains, on the other hand, are inherently infinite both in time and space, and moreover they are only partially observable. In this article we suggest a systematic method whose motivation comes from the desire to transform the task-to-be-solved into a finite-state, discrete-time, "approximately" Markovian task, which is completely observable too. The key idea is to break up the problem into subtasks and design controllers for each of the subtasks. Then operating conditions are attached to the controllers (together the controllers and their operating conditions which are called modules) and possible additional features are designed to facilitate observability. A new discrete time-counter is introduced at the "module-level" that clicks only when a change in the value of one of the features is observe...

We present an architecture for autonomous creatures that allows learning to be combined with action selection, based on ideas from ethology. We show how temporal-difference learning may be used within the context of an ethologically inspired animat architecture to build and modify portions of the behavior network, and to set fundamental parameters including the strength associated with individual Releasing Mechanisms, the time course associated with appetitive behaviors, and the learning rates to be used based on the observed reliability of specific contingencies. The learning algorithm has been implemented as part of the Hamsterdam toolkit for building autonomous animated creatures. When implemented in Silas, a virtual dog, the algorithm enables Silas to be trained using classical and instrumental conditioning. 1 Introduction Action selection and learning represent two significant areas of research in behavior-based AI [Maes94], and advances in both areas are essential if we are to ...

Computer programming environments for learning should make it easy to create worlds of responsive and autonomous objects, such as video games or simulations of animal behavior. But building such worlds remains difficult, partly because the models and metaphors underlying traditional programming languages are not particularly suited to the task. This dissertation investigates new metaphors, environments, and languages that make possible new ways to create programs -- and, more broadly, new ways to think about programs. In particular, it introduces the idea of programming with "agents" as a means to help people create worlds involving responsive, interacting objects. In this context, an agent is a simple mechanism intended to be understood through anthropomorphic metaphors and endowed with certain lifelike properties such as autonomy, purposefulness, and emotional state. Complex behavior is achieved by combining simple agents into more complex structures. While the agent metaphor enables...

The goal of our work aims at implementing progressively an action selection affective model for virtual humans that should be in the end autonomous, adaptive and sociable. Affect, traditionally distinguished from "cold " cognition, regroups emotions and motivations which are highly intertwined. We present a bottom-up approach by implementing first a motivational model of action selection to obtain motivationally autonomous virtual humans. For the adaptability of virtual humans and completeness of our affective model of action selection, we will define the interactions between motivations and emotions in order to integrate an emotional layer. In order to understand how they affect decision making in virtual humans, the motivations should represent more quantitative aspect of the decision making whereas emotions should be more qualitative one. 1

One of the defining features of intelligent behaviour is the ordering of individual expressed actions into coherent, apparently rational patterns. Psychology has long assumed that hierarchical and sequential structures internal to the intelligent agent underlie this expression. Recently these assumptions have been challenged by claims that behaviour controlled by such structures is necessarily rigid, brittle, and incapable of reacting quickly and opportunistically to changes in the environment (Hendriks-Jansen 1996, Goldfield 1995, Brooks 1991a). This dissertation is intended to support the hypothesis that sequential and hierarchical structures are necessary to intelligent behaviour, and to refute the above claims of their impracticality. Three forms of supporting evidence are provided: • a demonstration in the form of experimental results in two domains that structured intelligence can lead to robust and reactive behaviour, • a review of recent research results and paradigmatic trends within artificial intelligence, and • a similar examination of related research in natural intelligence.

Behavior selection is typically a “built-in” feature of behavior-based architectures and hence not amenable to change. There are, however, circumstances where changing behavior selection strategies is useful and can lead to better performance. In this paper, we demonstrate that such dynamic changes of behavior selection mechanisms are beneficial in several circumstances. We first categorize existing behavior selection mechanisms along three dimensions and then discuss seven possible circumstances where dynamically switching among them can be beneficial. Using the agent architecture framework APOC, we show how instances of all (non-empty) categories can be captured and how additional architectural mechanisms can be added to allow for dynamic switching among them. In particular, we propose a generic architecture for dynamic behavior selection, which can integrate existing behavior selection mechanisms in a unified way. Based on this generic architecture, we then verify that dynamic behavior selection is beneficial in the seven cases by defining architectures for simulated and robotic agents and performing experiments with them. The quantitative and qualitative analyses of the results obtained from extensive simulation studies and experimental runs with robots verify the utility of the proposed mechanisms.