Results 1 - 10
of
67
The Open Agent Architecture: A Framework for Building Distributed Software Systems
, 1999
"... The Open Agent Architecture #OAA#, developed and used for several years at SRI International, makes it possible for software services to be provided through the cooperative e#orts of distributed collections of autonomous agents. Communication and cooperation between agents are brokered by one or mor ..."
Abstract
-
Cited by 300 (9 self)
- Add to MetaCart
The Open Agent Architecture #OAA#, developed and used for several years at SRI International, makes it possible for software services to be provided through the cooperative e#orts of distributed collections of autonomous agents. Communication and cooperation between agents are brokered by one or more facilitators, which are responsible for matching requests, from users and agents, with descriptions of the capabilities of other agents. Thus, it is not generally required that a user or agent know the identities, locations, or number of other agents involved in satisfying a request. OAA is structured so as to minimize the e#ort involved in creating new agents and #wrapping" legacy applications, written in various languages and operating on various platforms; to encourage the reuse of existing agents; and to allow for dynamism and #exibilityin the makeup of agent communities. Distinguishing features of OAA as compared with related work include extreme #exibility in using fac...
QuickSet: Multimodal Interaction for Distributed Applications
, 1997
"... This paper presents an emerging application of multimodal interface research to distributed applications. We have developed the QuickSet prototype, a pen/voice system running on a hand-held PC, communicating via wireless LAN through an agent architecture to a number of systems, including NRaD's ..."
Abstract
-
Cited by 289 (35 self)
- Add to MetaCart
This paper presents an emerging application of multimodal interface research to distributed applications. We have developed the QuickSet prototype, a pen/voice system running on a hand-held PC, communicating via wireless LAN through an agent architecture to a number of systems, including NRaD's LeatherNet system, a distributed interactive training simulator built for the US Marine Corps. The paper describes the overall system architecture, a novel multimodal integration strategy offering mutual compensation among modalities, and provides examples of multimodal simulation setup. Finally, we discuss our applications experience and evaluation.
Designing the User Interface for Multimodal Speech and Pen-based Gesture Applications: State-of-the-Art Systems and Future Research Directions
, 2000
"... The growing interest in multimodal interface design is inspired in large part by the goals of supporting more transparent, flexible, efficient, and powerfully expressive means of humancomputer interaction than in the past. Multimodal interfaces are expected to support a wider range of diverse applic ..."
Abstract
-
Cited by 150 (15 self)
- Add to MetaCart
The growing interest in multimodal interface design is inspired in large part by the goals of supporting more transparent, flexible, efficient, and powerfully expressive means of humancomputer interaction than in the past. Multimodal interfaces are expected to support a wider range of diverse applications, to be usable by a broader spectrum of the average population, and to function more reliably under realistic and challenging usage conditions. In this paper, we summarize the emerging architectural approaches for interpreting speech and pen-based gestural input in a robust manner--- including early and late fusion approaches, and the new hybrid symbolic/statistical approach. We also describe a diverse collection of state-of-the-art multimodal systems that process users' spoken and gestural input. These applications range from map-based and virtual reality systems for engaging in simulations and training, to field medic systems for mobile use in noisy environments, to web-based transactions and standard text-editing applications that will reshape daily computing and have a significant commercial impact. To realize successful multimodal systems of the future, many key research challenges remain to be addressed. Among these challenges are the development of cognitive theories to guide multimodal system design, and the development of effective natural language processing, dialogue processing, and error handling techniques. In addition, new multimodal systems will be needed that can function more robustly and adaptively, and with support for collaborative multi-person use. Before this new class of systems can proliferate, toolkits also will be needed to promote software development for both simulated and functioning systems. Multimodal Speech and Gesture Interfaces 3 CONT...
Unification-based Multimodal Integration
- In Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics
, 1997
"... Recent empirical research has shown conclusive advantages of multimodal interaction over speech-only interaction for mapbased tasks. This paper describes a multimodal language processing architecture which supports interfaces allowing simultaneous input from speech and gesture recognition. Integrati ..."
Abstract
-
Cited by 112 (23 self)
- Add to MetaCart
Recent empirical research has shown conclusive advantages of multimodal interaction over speech-only interaction for mapbased tasks. This paper describes a multimodal language processing architecture which supports interfaces allowing simultaneous input from speech and gesture recognition. Integration of spoken and gestural input is driven by unification of typed feature structures representing the semantic contributions of the different modes. This integration method allows the component modalities to mutually compensate for each others' errors. It is implemented in Quick- Set, a multimodal (pen/voice) system that enables users to set up and control dis tributed interactive simulations.
Distributed mediation of ambiguous context in aware environments
- In Proceedings of the 15th annual ACM symposium on User interface software and technology
, 2002
"... Many context-aware services make the assumption that the context they use is completely accurate. However, in reality, both sensed and interpreted context is often ambiguous. A challenge facing the development of realistic and deployable context-aware services, therefore, is the ability to handle am ..."
Abstract
-
Cited by 42 (5 self)
- Add to MetaCart
(Show Context)
Many context-aware services make the assumption that the context they use is completely accurate. However, in reality, both sensed and interpreted context is often ambiguous. A challenge facing the development of realistic and deployable context-aware services, therefore, is the ability to handle ambiguous context. In this paper, we describe an architecture that supports the building of context-aware services that assume context is ambiguous and allows for mediation of ambiguity by mobile users in aware environments. We illustrate the use of our architecture and evaluate it through three example contextaware services, a word predictor system, an In/Out Board, and a reminder tool.
Multimodal user interfaces in the open agent architecture
- Proceedings of the 1997 International Conference on Intelligent User Interfaces
, 1997
"... ABSTRACT The design and development of the Open Agent Architecture (OAA) 1 system has focused on providing access to agentbased applications through an intelligent, cooperative, distributed, and multimodal agent-based user interfaces. The current multimodal interface supports a mix of spoken lang ..."
Abstract
-
Cited by 39 (8 self)
- Add to MetaCart
(Show Context)
ABSTRACT The design and development of the Open Agent Architecture (OAA) 1 system has focused on providing access to agentbased applications through an intelligent, cooperative, distributed, and multimodal agent-based user interfaces. The current multimodal interface supports a mix of spoken language, handwriting and gesture, and is adaptable to the user's preferences, resources and environment. Only the primary user interface agents need run on the local computer, thereby simplifying the task of using a range of applications from a variety of platforms, especially low-powered computers such as Personal Digital Assistants (PDAs). An important consideration in the design of the OAA was to facilitate mix-andmatch: to facilitate the reuse of agents in new and unanticipated applications, and to support rapid prototyping by facilitating the replacement of agents by better versions. The utility of the agents and tools developed as part of this ongoing research project has been demonstrated by their use as infrastructure in unrelated projects. Keywords: agent architecture, multimodal, speech, gesture, handwriting, natural language INTRODUCTION A major component of our research on multiagent systems is in the user interface to large communities of agents. We have developed agent-based multimodal user interfaces using the same agent architecture used to build the back ends of these applications. We describe these interfaces and the larger architecture, and outline some of the applications that have been built using this architecture and interface agents.
A Unified Framework for Constructing Multimodal Experiments and Applications
- Conference on Cooperative Multimodal Communication (CMC98
, 1998
"... . In 1994, inspired by a Wizard of Oz (WOZ) simulation experiment, we developed a working prototype of a system that enables users to interact with a map display through synergistic combinations of pen and voice. To address many of the issues raised by multimodal fusion, our implementation employed ..."
Abstract
-
Cited by 28 (14 self)
- Add to MetaCart
. In 1994, inspired by a Wizard of Oz (WOZ) simulation experiment, we developed a working prototype of a system that enables users to interact with a map display through synergistic combinations of pen and voice. To address many of the issues raised by multimodal fusion, our implementation employed a distributed multiagent framework to coordinate parallel competition and cooperation among processing components. Since then, the agent-based infrastructure has been enhanced with a collaboration technology, creating a framework in which multiple humans and automated agents can naturally interact within the same graphical workspace. Our current endeavor is the leveraging of this architecture to create a unified implementation framework for simultaneously developing both WOZ simulated systems and their fully automated counterparts. Bootstrapping effects made possible by such an approach are illustrated by a hybrid WOZ experiment currently under way in our laboratory: as a naive subject draws...
Cognitive status and form of reference in multimodal human-computer interaction
- In AAAI
"... We analyze a corpus of referring expressions collected from user interactions with a multimodal travel guide application. The analysis suggests that, in dramatic contrast to normal modes of human-human interaction, the interpretation of referring expressions can be computed with very high accuracy u ..."
Abstract
-
Cited by 22 (0 self)
- Add to MetaCart
(Show Context)
We analyze a corpus of referring expressions collected from user interactions with a multimodal travel guide application. The analysis suggests that, in dramatic contrast to normal modes of human-human interaction, the interpretation of referring expressions can be computed with very high accuracy using a model which pairs an impoverished notion of discourse state with a simple set of rules that are insensitive to the type of referring expression used. We attribute this result to the implicit manner in which the interface conveys the system’s beliefs about the operative discourse state, to which users tailor their choice of referring expressions. This result offers new insight into the way computer interfaces can shape a user’s language behavior, insights which can be exploited to bring otherwise difficult interpretation problems into the realm of tractability.
MVIEWS: Multimodal Tools for the Video Analyst
- In Proceedings of the 1998 International Conference on Intelligent User Interfaces (IUI98
, 1998
"... Full-motion video has inherent advantages over still imagery for characterizing events and movement. Military and intelligence analysts currently view live video imagery from airborne and ground-based video platforms, but few tools exist for efficient exploitation of the video and its accompanying m ..."
Abstract
-
Cited by 19 (5 self)
- Add to MetaCart
(Show Context)
Full-motion video has inherent advantages over still imagery for characterizing events and movement. Military and intelligence analysts currently view live video imagery from airborne and ground-based video platforms, but few tools exist for efficient exploitation of the video and its accompanying metadata. In pursuit of this goal, SRI has developed MVIEWS, a system for annotating, indexing, extracting, and disseminating information from video streams for surveillance and intelligence applications. MVIEWS is implemented within the Open Agent Architecture, a distributed multiagent framework that enables rapid integration of component technologies; for MVIEWS, these technologies include pen and voice recognition and interpretation, image processing and object tracking, geo-referenced interactive maps, multimedia databases, and human collaborative tools. Keywords Multimodal pen and voice user interfaces, image processing and object tracking, video analysis and annotation, agent architec...
