In this paper a simulation model for visual attention is discussed and formally analysed. The model is part of the design of an agent-based system that supports a naval officer in its task to compile a tactical picture of the situation in the field. A case study is described in which the model is used to simulate a human subject’s attention. The formal analysis is based on temporal relational specifications for attentional states and for different stages of attentional processes. The model has been automatically verified against these specifications.

Abstract. A reusable agent-based generic model is presented for a specific class of Ambient Intelligence applications: those cases addressing human wellbeing and functioning from a human-like understanding. The model incorporates ontologies, knowledge and dynamic models from human-directed sciences such as psychology, social science, neuroscience and biomedical sciences. The model has been formally specified, and it is shown how for specific applications it can be instantiated by application-specific elements, thus providing an executable specification that can be used for prototyping. Moreover, it is shown how dynamic properties can be formally specified and verified against generated traces. 1

Abstract. To improve the performance and wellbeing of humans in complex human-computer interaction settings, an interesting challenge for an ambient (or pervasive) agent system is to recognise the emotions of humans. To this end, this paper introduces a computational model to estimate the process of emotion generation based on certain triggers. The model has been implemented and tested using the modelling language LEADSTO. A first evaluation indicates that the model is successful in estimating a person’s emotions, and is robust to different parameter settings. 1

Abstract. In this paper a system for visual attention manipulation is introduced and formally described. This system is part of the design of a software agent that supports naval crew in her task to compile a tactical picture of the situation in the field. A case study is described in which the system is used to manipulate a human subject’s attention. To this end the system includes a Theory of Mind about human attention and uses this to estimate the subject’s current attention, and to determine how features of displayed objects have to be adjusted to make the attention shift in a desired direction. Manipulation of attention is done by adjusting illumination according to the calculated difference between a model describing the subject’s attention and a model prescribing it. 1

Abstract. This paper addresses the development of an adaptive cooperative agent in a domain that suffers from human error in the allocation of attention. The design is discussed of a component of this adaptive agent, called Human Attention-Based Task Allocator (HABTA), capable of managing agent and human attention. The HABTA-component reallocates the human’s and agent’s focus of attention to tasks or objects based on an estimation of the current human allocation of attention and by comparison of this estimation with certain normative rules. The main contribution of the present paper is the description of the combined approach of design and validation for the development of such components. Two complementary experiments of validation of HABTA are described. The first experiment validates the model of human attention that is incorporated in HABTA, comparing estimations of the model with those of humans. The second experiment validates the HABTA-component itself, measuring its effect in terms of human-agent team performance, trust, and reliance. Finally, some intermediary results of the first experiment are shown, using human data in the domain of naval warfare. 1

Abstract. This paper discusses the use of cognitive models as augmented metacognition on task allocation for tasks requiring visual attention. In the domain of naval warfare, the complex and dynamic nature of the environment makes that one has to deal with a large number of tasks in parallel. Therefore, humans are often supported by software agents that take over part of these tasks. However, a problem is how to determine an appropriate allocation of tasks. Due to the rapidly changing environment, such a work division cannot be fixed beforehand: dynamic task allocation at runtime is needed. Unfortunately, in alarming situations the human does not have the time for this coordination. Therefore system-triggered dynamic task allocation is desirable. The paper discusses the possibilities of such a system for tasks requiring visual attention.

In this paper an analysis of the notion of attention is described that comprises a differentiation according to five stages of an attentional process related to action selection and performance. The first stage deals with the allocation of attention to external and internal stimuli, the second with examination and analysis, the third with decision making and action selection, the fourth with action preparation and performance, and the fifth with action assessment. The analysis involves temporal formalisation and validation based on data from a human operator executing a warfare task.

A reusable agent-based generic model is presented for a specific class of Ambient Intelligence applications: those cases addressing human wellbeing and functioning from a human-like understanding. The model incorporates ontologies, knowledge and dynamic models from human-directed sciences such as psychology, social science, neuroscience and biomedical sciences. The model has been formally specified, and it is shown how for specific applications it can be instantiated by application-specific elements, thus providing an executable specification that can be used for prototyping. Moreover, it is shown how dynamic properties can be formally specified and verified against generated traces.

In the domain of naval warfare, it is crucial for the crew of the vessels involved to be aware of the situation in the field. Examples of important questions that should be addressed continuously are “in which direction are we heading?”, “are we currently under attack?”, “are there any friendly vessels around?”, and so on. To assess such issues, one of the crew members is usually assigned the Tactical Picture Compilation Task (TPCT): the task to identify and classify all entities in the environment. This is done by monitoring a radar screen for radar contacts, and reasoning with the available information in order to determine the type and intent of the contacts on the screen. However, due to the complex and dynamic nature of the environment, this person has to deal with a large number of tasks in parallel. Often the radar contacts are simply too numerous and dynamic to be adequately monitored by a single human, which compromises the performance of the task. For these reasons, it may be useful to offer the human some support from an intelligent ambient system, consisting of software agents that assist him in the execution of the Tactical Picture Compilation Task. For example, in case the human is directing its attention on the left part of a radar screen, but ignores an important contact that just entered the radar screen from the right, such an agent may alert him about the arrival of that new contact. To be able to provide this kind of intelligent support, the system somehow needs to maintain

In this chapter, we propose to outline the scientific area that addresses Ambient Intelligence applications in which not only sensor data, but also knowledge from the human-directed sciences such as biomedical science, neuroscience, and psychological and social sciences is incorporated. This knowledge enables the environment to perform more in-depth, human-like analyses of the functioning of the observed humans, and to come up with better informed actions. A structured approach to embed human knowledge in Ambient Intelligence applications is presented an illustrated using two examples, one on automated visual attention manipulation, and another on the assessment of the behaviour of a car driver.