We present a new abductive, probabilistic theory of plan recognition. This model differs from previous theories in being centered around a model of plan execution: most previous methods have been based on plans as formal objects or on rules describing the recognition process. We show that our new model accounts for phenomena omitted from most previous plan recognition theories: notably the cumulative effect of a sequence of observations of partially-ordered, interleaved plans and the effect of context on plan adoption. The model also supports inferences about the evolution of plan execution in situations where another agent intervenes in plan execution. This facility provides support for using plan recognition to build systems that will intelligently assist a user. 1

this paper was supported by a gift from the Systems Development Foundation. The research was done as part of my doctoral thesis [19], which was supported by a gift from the Systems Development Foundation, by an IBM Graduate Fellowship, by the Defense Advanced Research Projects Agency under Contract N00039-84-K-0078 and by the Office of Naval Research under Contract N00014-855-C-0013. My sincere thanks to Barbara Grosz, and to all the others, too numerous to list here, who contributed to my thesis effort.

cockpit interface Abstract cockpit interface Abstract cockpit interface Abstract cockpit interface Figure 1: Human and automated pilots interact with the DIS environment via distributed simulators.

. The paper is concerned with the succinct axiomatization and ecient deduction of non-change, within McCarthy and Hayes' Situation Calculus. The idea behind the proposed approach is this: suppose that in a room containing a man, a robot and a cat as the only potential agents, the only action taken by the man within a certain time interval is to walk from one place to another, while the robot's only actions are to pick up a box containing the (inactive) cat and carry it from its initial place to another. We wish to prove that a certain object (such as the cat, or the doormat) did not change color. We reason that the only way it could have changed color is for the man or the robot to have painted or dyed it. But since these are not among the actions which actually occurred, the color of the object is unchanged. Thus we need no frame axioms to the eect that walking and carrying leave colors unchanged (which is in general false in multi-agent worlds), and no default schema that properties change only when we can prove they do (which is in general false in incompletely known worlds). Instead we use explanationclosure axioms specifying all primitive actions which can produce a given type of change within the setting of interest. A method similar to this has been proposed by Andrew Haas for singleagent, serial worlds. The contribution of the present paper lies in 1 showing (1) that such methods do indeed encode non-change succinctly, (2) are independently motivated, (3) can be used to justify highly ecient methods of inferring non-change, specically the \sleeping dog" strategy of STRIPS, and (4) can be extended to simple multiagent worlds with concurrent actions. An ultimate limitation may lie in the lack of a uniform strategy for deciding what ...

■ We describe an approach to intelligent user interfaces, based on the idea of making the computer a collaborator, and an application-independent technology for implementing such interfaces. What properties of a user interface would make you want to call it intelligent? For us, any interface that is called intelligent should at least be able to answer the six types of questions from users shown in figure 1. Being able to ask and answer these kinds of questions implies a flexible and adaptable division of labor between the human and the computer in the interaction process. Unlike most current interfaces, an intelligent user interface should be able to

In this paper, we present a method for recognising an agent's behaviour in dynamic, noisy, uncertain domains, and across multiple levels of abstraction. We term this problem on-line plan recognition under uncertainty and view it generally as probabilistic inference on the stochastic process representing the execution of the agent's plan. Our contributions in this paper are twofold. In terms of probabilistic inference, we introduce the Abstract Hidden Markov Model (AHMM), a novel type of stochastic processes, provide its dynamic Bayesian network (DBN) structure and analyse the properties of this network. We then describe an application of the Rao-Blackwellised Particle Filter to the AHMM which allows us to construct an ecient, hybrid inference method for this model. In terms of plan recognition, we propose a novel plan recognition framework based on the AHMM as the plan execution model. The Rao-Blackwellised hybrid inference for AHMM can take advantage of the independence properties inherent in a model of plan execution, leading to an algorithm for online probabilistic plan recognition that scales well with the number of levels in the plan hierarchy. This illustrates that while stochastic models for plan execution can be complex, they exhibit special structures which, if exploited, can lead to efficient plan recognition algorithms. We demonstrate the usefulness of the AHMM framework via a behaviour recognition system in a complex spatial environment using distributed video surveillance data.

this paper, is when two participants can both communicate with each other and observe each other's actions on some shared artifact

This paper presents a computational model of how conversational participants collaborate in making referring expressions. The model is based on the planning paradigm. It employs plans for constructing and recognizing referring expressions and meta-plans for constructing and recognizing clarifications. This allows the model to account for the generation and understanding both of referring expressions and of their clarifications in a uniform framework using a single knowledge base.

In this paper we present results related to achieving finegrained activity recognition for context-aware computing applications. We examine the advantages and challenges of reasoning with globally unique object instances detected by an RFID glove. We present a sequence of increasingly powerful probabilistic graphical models for activity recognition. We show the advantages of adding additional complexity and conclude with a model that can reason tractably about aggregated object instances and gracefully generalizes from object instances to their classes by using abstraction smoothing. We apply these models to data collected from a morning household routine. 1.

The situation recognition system, to which this paper is devoted, receives as input a stream of time-stamped events; it performs recognition of instances of occurring situations, as they are developing, and it generates as output deduced events and actions to trigger. It is mainly a temporal reasoning system. It is predictive in the sense that it predicts forthcoming events relevant to its task, it focuses its attention on them and it maintains their temporal windows of relevance. Its main functionality is to recognize efficiently complex temporal patterns on the fly, while they are taking place. This system has been tested for the surveillance of an environment by a multisensory perception machine; it is being applied to monitoring a complex dynamic system. 1