This research investigates the cognitive strategies and eye movements that people use to search for a known item in a hierarchical computer display. Computational cognitive models were built to simulate the visual-perceptual and oculomotor processing required to search hierarchical and nonhierarchical displays. Eye movement data were collected and compared on over a dozen measures with the a priori predictions of the models. Though it is well accepted that hierarchical layouts are easier to search than nonhierarchical layouts, the underlying cognitive basis for this design heuristic has not yet been established. This work combines cognitive modeling and eye tracking to explain this and numerous other visual design guidelines. This research also demonstrates the power of cognitive modeling for predicting, explaining, and interpreting eye movement data, and how to use eye tracking data to confirm and disconfirm modeling details. Categories and subject descriptors H.5.2 [Information Interfaces and Presentation]: User Interfaces-- Evaluation/methodology, eye tracking,

We study a new task, proactive information retrieval by combining implicit relevance feedback and collaborative filtering. We have constructed a controlled experimental setting, a prototype application, in which the users try to find interesting scientific articles by browsing their titles. Implicit feedback is inferred from eye movement signals, with discriminative hidden Markov models estimated from existing data in which explicit relevance feedback is available. Collaborative filtering is carried out using the User Rating Profile model, a state-of-the-art probabilistic latent variable model, computed using Markov Chain Monte Carlo techniques. For new document titles the prediction accuracy with eye movements, collaborative filtering, and their combination was significantly better than by chance. The best prediction accuracy still leaves room for improvement but shows that proactive information retrieval and combination of many sources of relevance feedback is feasible.

This paper describes technology and tools for intelligent human-computer interaction (IHCI) where human cognitive, perceptual, motor, and affective factors are modeled and used to adapt the H--C interface. IHCI emphasizes that human behavior encompasses both apparent human behavior and the hidden mental state behind behavioral performance. IHCI expands on the interpretation of human activities, known as W4 (what, where, when, who). While W4 only addresses the apparent perceptual aspect of human behavior, the W5+ technology for IHCI described in this paper addresses also the why and how questions, whose solution requires recognizing specific cognitive states. IHCI integrates parsing and interpretation of nonverbal information with a computational cognitive model of the user, which, in turn, feeds into processes that adapt the interface to enhance operator performance and provide for rational decision-making. The technology proposed is based on a general four-stage interactive framework, which moves from parsing the raw sensory-motor input, to interpreting the user's motions and emotions, to building an understanding of the user's current cognitive state. It then diagnoses various problems in the situation and adapts the interface appropriately. The interactive component of the system improves processing at each stage. Examples of perceptual, behavioral, and cognitive tools are described throughout the paper. Adaptive and intelligent HCI are important for novel applications of computing, including ubiquitous and human-centered computing

We propose a head-free, easy-setup gaze tracking system designed for a gaze-based Human-Computer Interaction. Our system enables the user to interact with the computer soon after catching the user's eye blinks. The user can move his/her head freely since the system keeps tracking the user's eye. In addition, our system only needs a 10 second calibration procedure at the very first time of use. An eye tracking method based on our unique eye blink detection and a sophisticated gaze estimation method using the geometrical eyeball model realize these advantages.

We organize a PASCAL EU Network of Excellence challenge for inferring relevance from eye movements, beginning 1 March 2005. The aim of this paper is to provide background material for the competitors: give references to related articles on eye movement modelling, describe the methods used for extracting the features used in the challenge, provide results of basic reference methods and to discuss open questions in the field. 1

We study whether it is possible to infer from eye movements measured during reading what is relevant for the user in an information retrieval task. Inference is made using hidden Markov and discriminative hidden Markov models. The result of this feasibility study is that prediction of relevance is possible to a certain extent, and models benefit from taking into account the time series nature of the data.

While the collection of behavioral protocols has been common practice in human-computer interaction research for many years, the analysis of large protocol data sets is often extremely tedious and time-consuming, and automated analysis methods have been slow to develop. This paper proposes an automated method of protocol analysis to find canonical behaviors — a small subset of protocols that is most representative of the full data set, providing a reasonable “big picture ” view of the data with as few protocols as possible. The automated method takes advantage of recent algorithmic developments in computational vision, modifying them to allow for distance measures between behavioral protocols. The paper includes an application of the method to web-browsing protocols, showing how the canonical behaviors found by the method match well to sets of behaviors identified by expert human coders.

ISBN 951-22-8219-4 (printed version) ISBN 951-22-8220-8 (electronic version)

Eye-tracking has been employed in usability engineering for many years because, among other things, it affords usability practitioners information about where users focus their attention. It helps practitioners identify the extent to which the visual display elements presented on many interactive products enhance or detract from the user experience. Eye movement data offer system developers and usability engineers information about visual attention, visual search efficiency, and visual information processing while users interact with a system. In this study, we tracked participants ‘ eye movements as they viewed newspaper and TV-oriented news Websites. We used several visual attention measures (number of fixations, fixation duration, gaze time, and saccade rate) and scan path analysis to investigate whether ocular behavior differed by type of news site. We found that newspaper and TV-oriented site types did not influence measures of visual attention. However, the areas where participants fixated differed by site type. In addition, there was greater across-user variability in the viewing of newspaper homepages compared to TV homepages. Finally, we report on the utility of examining visual attention using scan path analysis and string-editing methods. These methods were especially useful for identifying fixation areas as well as variability in participants ‘ scan paths.

Abstract. The collection of behavior protocols is a common practice in human factors research, but the analysis of these large data sets has always been a tedious and time-consuming process. We are interested in automatically finding canonical behaviors: a small subset of behavioral protocols that is most representative of the full data set, providing a view of the data with as few protocols as possible. Behavior protocols often have a natural graph-based representation, yet there has been little work applying graph theory to their study. In this paper we extend our recent algorithm by taking into account the graph topology induced by the paths taken through the space of possible behaviors. We applied this technique to find canonical web-browsing behaviors for computer users. By comparing identified canonical sets to a ground truth determined by expert human coders, we found that this graph-based metric outperforms our previous metric based on edit distance. 1