We study discriminative joint density models, that is, generative models for the joint density p(c, x) learned by maximizing a discriminative cost function, the conditional likelihood. We use the framework to derive generative models for generalized linear models, including logistic regression, linear discriminant analysis, and discriminative mixture of unigrams. The benefits of deriving the discriminative models from joint density models are that it is easy to extend the models and interpret the results, and missing data can be treated using justified standard methods.

Abstract. This paper explores the feasibility of using an eye tracker as an image retrieval interface. A database of image similarity values between 1000 Corel images is used in the study. Results from participants performing image search tasks show that eye tracking data can be used to reach target images in fewer steps than by random selection. The effects of the intrinsic difficulty of finding images and the time allowed for successive selections were also investigated. 1

We introduce GaZIR, a gaze-based interface for browsing and searching for images. The system computes on-line predictions of relevance of images based on implicit feedback, and when the user zooms in, the images predicted to be the most relevant are brought out. The key novelty is that the relevance feedback is inferred from implicit cues obtained in real-time from the gaze pattern, using an estimator learned during a separate training phase. The natural zooming interface can be connected to any content-based information retrieval engine operating on user feedback. We show with experiments on one engine that there is sufficient amount of information in the gaze patterns to make the estimated relevance feedback a viable choice to complement or even replace explicit feedback by pointing-and-clicking.

This paper explores the feasibility of using an eye tracker as an image retrieval interface. A database of image similarity values between 1000 Corel images is used in the study. Results from participants performing image search tasks show that eye tracking data can be used to reach target images quicker than by random selection. The effects of the intrinsic difficulty of finding images and the time allowed for successive selections were also investigated.

Abstract. Our goal in this study was to explore the potentials of extracting features from eye-tracking data that have the potential to improve performance in implicit relevance feedback. We view this type of data as an example of the searcher ’ immediate context and as containing useful clues of the indications of the interaction between the searcher and the IR system. In particular, we explored if we could qualitatively identify features have potential to improve performance in implicit relevance feedback, and how such features correlate with document elements assessed as relevant or non-relevant. The results point to so-called thorough reading as one of the most promising features for identifying relevant information as input for implicit relevance feedback – in particular when it is related to the total time the searcher has looked an element.

A long term goal in user modeling for improving human-computer interaction is to understand the user's intent based on her monitored actions. We are developing an information retrieval system where the task is to predict relevance for new documents, given judgments on old ones. By monitoring the user's eye movements and inferring implicit feedback from them we reduce the amount of tedious ranking of retrieved documents, called relevance feedback in standard information retrieval. Relevance is inferred with machine learning methods, trained on eye movement patterns measured in settings where relevance is known. Noise in the predictions is compensated for by fusing the eye movements with other information about the user's preferences. The goal is to make the information retrieval system proactive, that is, capable of anticipating the user's interests.

... and Interests), carried out during 2003–2005. The project focused on how to construct and combine user models from implicit or explicit feedback signals. If proper user models can be constructed, it will be possible to build proactive applications, that is, applications that learn to anticipate the user’s needs. Our prototype application is information retrieval, where implicit feedback signal is measured from eye movements. Relevance of read text is extracted from the feedback signal with hidden Markov models learned from a collected data set. Since relevance in general is hard to define, we have constructed an experimental setting where relevance is known a priori. The implicit feedback signal is very noisy. Thus, it needs to be supplemented with relevance predictions from other available sources. In the prototype application an alternative relevance prediction was obtained from collaborative filtering. For new document titles the prediction accuracy with eye movements, collaborative filtering, and their combination was significantly better than by chance. The best prediction accuracy still leaves room for improvement but shows that proactive information retrieval and combination of many sources of relevance feedback is feasible.

Abstract not found

Abstract We tackle the problem of new users or documents in collaborative filtering. Generalization over users by grouping them into user groups is beneficial when a rating is to be predicted for a relatively new document having only few observed ratings. Analogously, generalization over documents improves predictions in the case of new users. We show that if either users and documents or both are new, two-way generalization becomes necessary. We demonstrate the benefits of grouping of users, grouping of documents, and two-way grouping, with artificial data and in two case studies with real data. We have introduced a probabilistic latent grouping model for predicting the relevance of a document to a user. The model assumes a latent group structure for both users and items. We compare the model against a state-of-the-art method, the User Rating Profile model, where only the users have a latent group structure. We compute the posterior of both models by Gibbs sampling. The Two-Way Model predicts relevance more accurately when the target consists of both new documents and new users. The reason is that generalization over documents becomes beneficial for new documents and at the same time generalization over users is needed for new users.

We have developed a prototype platform for contextual information access in mobile settings. Objects, people, and the environment are considered as contextual channels or cues to more information. The system infers, based on gaze, speech and other implicit feedback signals, which of the contextual cues are relevant, retrieves more information relevant to the cues, and presents the information with Augmented Reality (AR) techniques on a handheld or headmounted display. The augmented information becomes potential contextual cues as well, and its relevance is assessed to provide more information. In essence, the platform turns the real world into an information browser which focuses proactively on the information inferred to be the most relevant for the user. We present the first pilot application, a Virtual Laboratory Guide, and its early evaluation results. 1.