Abstract. We present a comparative analysis of a layered architecture of Hidden Markov Models (HMMs) and dynamic Bayesian networks (DBNs) for identifying human activites from multimodal sensor information. We use the two representations to diagnose users ’ activities in S-SEER, a multimodal system for recognizing office activity from realtime streams of evidence from video, audio and computer (keyboard and mouse) interactions. As the computation required for sensing and processing perceptual information can impose significant burdens on personal computers, the system is designed to perform selective perception using expected-value-of-information (EVI) to limit sensing and analysis. We discuss the relative performance of HMMs and DBNs in the context of diagnosis and EVI computation. 1

Abstract—Many information fusion applications are often characterized by a high degree of complexity because: 1) data are often acquired from sensors of different modalities and with different degrees of uncertainty; 2) decisions must be made efficiently; and 3) the world situation evolves over time. To address these issues, we propose an information fusion framework based on dynamic Bayesian networks to provide active, dynamic, purposive and sufficing information fusion in order to arrive at a reliable conclusion with reasonable time and limited resources. The proposed framework is suited to applications where the decision must be made efficiently from dynamically available information of diverse and disparate sources. Index Terms—Active sensing, Bayesian networks, information fusion. I.

We present four perceptual user interface systems that explore a continuum from explicit to implicit interaction. Explicit interactions include most of today’s mouse and keyboard-based interaction models, where the user initiates a discrete action and expects a timely discrete response. Implicit interactions may use passive monitoring of the user over longer periods of time, and result in changing some aspect of the rest of the interaction. For example, less urgent notifications may be withheld from the user if the system detects they are engaged in a meeting. The first system is FlowMouse, a program that tries to emulate the mouse but suggests more implicit kinds of interaction. Second, we describe GWindows, which focuses complementing the mouse in today’s GUI in a way that might support casual interactions. Then, ToughtLight eschews the traditional notion of an explicit discrete pointer altogether and in so doing presents a number of challenges in designing applications. And finally, S-SEER supports a purely implicit style of interaction driven by models of situational awareness. In presenting this series of projects in order from explicit to implicit modes, we hope to illustrate by way of example the various challenges and opportunities for perceptual user interfaces. 1

Abstract not found

We show how a greedy approach to visual search — i.e., directly moving to the most likely location of the target — can be suboptimal, if the target object is hard to detect. Instead it is more efficient and leads to higher detection accuracy to first look for other related objects, that are easier to detect. These provide contextual priors for the target that make it easier to find. We demonstrate this in simulation using POMDP models, focussing on two special cases: where the target object is contained within the related object, and where the target object is spatially adjacent to the related object.

Abstract—As sensors become more complex and prevalent, they present their own issues of cost effectiveness and timeliness. It becomes increasingly important to select sensor sets that provide the most information at the least cost and in the most timely and efficient manner. Two typical sensor selection problems appear in a wide range of applications. The first type involves selecting a sensor set that provides the maximum information gain within a budget limit. The other type involves selecting a sensor set that optimizes the tradeoff between information gain and cost. Unfortunately, both require extensive computations due to the exponential search space of sensor subsets. This paper proposes efficient sensor selection algorithms for solving both of these sensor selection problems. The relationships between the sensors and the hypotheses that the sensors aim to assess are modeled with Bayesian networks, and the information gain (benefit) of the sensors with respect to the hypotheses is evaluated by mutual information. We first prove that mutual information is a submodular function in a relaxed condition, which provides theoretical support for the proposed algorithms. For the budget-limit case, we introduce a greedy algorithm that has a constant factor of (1 − 1/e) guarantee to the optimal performance. A partitioning procedure is proposed to improve the computational efficiency of the algorithms by efficiently computing mutual information as well as reducing the search space. For the optimal-tradeoff case, a submodular–supermodular procedure is exploited in the proposed algorithm to choose the sensor set that achieves the optimal tradeoff between the benefit and cost in a polynomial-time complexity. Index Terms—Active fusion, Bayesian networks (BNs), sensor selection, submodular function.

Pervasive computing proposes that in the future, human beings will be immersed in a technology rich environment, where computing power will be embedded in devices all around us. This thesis examines how such technology rich environments can be effectively governed so that they meet the demands and expectations of the users that they will support. This examination is based around identifying and supporting three stakeholder groups. These groups reflect three different types of governance that needs to be exercised, namely: users who interact with the environment and expect support; administrators who need to model resources; and experts who can describe the types of routine behaviour that happen within the Pervasive Computing environment. The approach to providing this governance is based around developing an integrated set of services that individually satisfy the needs of a particular stakeholder, while also interacting to provide an overall platform for usercentric governance. Each of these services is user-centric and non-application specific. In so doing the tools cater to the needs of a variety of users, irrespective of their technical expertise or the application that the environment will support. The services have been evaluated experimentally and each has been compared with related work. This thesis articulates the design objectives and implementations that were used to produce the experimental prototypes, and it also offers an in-depth examination of the state of the art in a number of disciplines including Pervasive Computing, Event Aggregation, Mixed Initiative, Ontologies and Autonomic Computing.

We are interested in speeding up approximate inference in Markov Random Fields (MRFs). We present a new method that uses gates—binary random variables that determine which factors of the MRF to use. Which gates are open depends on the observed evidence; when many gates are closed, the MRF takes on a sparser and faster structure that omits “unnecessary ” factors. We train parameters that control the gates, jointly with the ordinary MRF parameters, in order to locally minimize an objective that combines