The growing interest in multimodal interface design is inspired in large part by the goals of supporting more transparent, flexible, efficient, and powerfully expressive means of humancomputer interaction than in the past. Multimodal interfaces are expected to support a wider range of diverse applications, to be usable by a broader spectrum of the average population, and to function more reliably under realistic and challenging usage conditions. In this paper, we summarize the emerging architectural approaches for interpreting speech and pen-based gestural input in a robust manner--- including early and late fusion approaches, and the new hybrid symbolic/statistical approach. We also describe a diverse collection of state-of-the-art multimodal systems that process users' spoken and gestural input. These applications range from map-based and virtual reality systems for engaging in simulations and training, to field medic systems for mobile use in noisy environments, to web-based transactions and standard text-editing applications that will reshape daily computing and have a significant commercial impact. To realize successful multimodal systems of the future, many key research challenges remain to be addressed. Among these challenges are the development of cognitive theories to guide multimodal system design, and the development of effective natural language processing, dialogue processing, and error handling techniques. In addition, new multimodal systems will be needed that can function more robustly and adaptively, and with support for collaborative multi-person use. Before this new class of systems can proliferate, toolkits also will be needed to promote software development for both simulated and functioning systems. Multimodal Speech and Gesture Interfaces 3 CONT...

This article examines the issue of developing semantics for agent communication languages. In particular, it considers the problem of giving a verifiable semantics for such languages -- a semantics where conformance (or otherwise) to the semantics could be determined by an independent observer. These problems are precisely dened in an abstract formal framework. Using this framework, a number of example agent communication frameworks are defined. A discussion is then presented, of the various options open to designers of agent communication languages, with respect the problem of verifying conformance.

This paper describes an implementation of some key aspects of a theory of dialogue processing whose main concerns are to provide models of GROUNDING and of the role of DISCOURSE OBLIGATIONS in an agent's deliberation processes. Our system uses the TrindiKit dialogue move engine toolkit, which assumes a model of dialogue in which a participant's knowledge is characterised in terms of INFORMATION STATES which are subject to various kinds of updating mechanisms.

We present an effective generic communicating rational agent, ARTIMIS, and its application to cooperative spoken dialogue. ARTIMIS ' kernel is the implementation of a formal theory of interaction. This theory involves a set of generic axioms which models, in a homogeneous logical framework, principles of rational behaviour, communication, and cooperation. The theory is interpreted by a specifically designed reasoning engine. When applied to the context of natural dialogue, ARTIMIS includes specialised components for speech and natural language processing. 1

Abstract. In this paper we describe an approach to the construction of adaptive tutoring systems, based on techniques from the research area of Reasoning about Actions and Change. This approach leads to the implementation of a prototype system, having a multi-agent architecture, whose kernel is a set of rational agents, programmed in the logic programming language DyLOG. In the prototype that we implemented the reasoning capabilities of the agents are exploited both to dynamically build study plans and to verify the correctness of user-given study plans with respect to the competence that the user wants to acquire.

This paper examines the issue of developing semantics for agent communication languages. In particular, it considers the problem of giving a verifiable semantics for such languages -- a semantics where conformance (or otherwise) to the semantics could be determined by an independent observer. These problems are precisely defined in an abstract formal framework. Using this framework, a number of example agent communication frameworks are examined with respect to the problem of verifying conformance. A discussion is then presented, of the various options open to designers of agent communication languages, with respect the problem of verifying conformance.

Abstract. In this paper we present a formal framework for social agents. The social agents consist of four components: the information component (containing knowledge and belief), the action component, the motivational component (where goals, intentions, etc. play arole) and the social component (containing aspects of speech acts and relations between agents). The main aim of this work was to describe all componentsin a uniform way, such that it is possible to verify each component separately but also formally describe the interactions between the different components. E.g. the effect of a speech act on the believes of an agent or on the commitment to a goal it pursues. 1

this paper we propose a combination of the dynamic logic programming paradigm and (a version of) KS-agents. In the resulting framework, rational, reactive agents can dynamically change their own knowledge bases as well as their own goals. In particular, at every iteration of an observe-think-act cycle, the agent can make observations, learn new facts and new rules from the environment, and then it can update its knowledge accordingly.

There is currently a broad interest in dialogue acts and dialogue act taxonomies, and new uses, taxonomies, and standardization efforts continue to be proposed. This paper presents a discussion of issues that must be addressed in order to facilitate the shared understanding and use of taxonomies. The discussion is framed in terms of 20 questions, the answers to which will help make the meanings of taxonomy elements more clear to different communities of users.

Abstract. We propose an approach to reasoning about conversation protocols within the framework of a logic-based agent language. We show how to embed a theory of communicative actions in the framework of a modal logic of action and beliefs, to specify software agents that, situated in a multi-agent environment, can interact with one another by a speech act based communication mechanism. Agents have their own local beliefs on the world and on the other agents mental state. Complex communicative behaviors can be specified as conversation protocols, and agents can reason on the belief dynamics caused by communications, before committing to a given interaction. 1 Introduction and