With more and more digital videos found online, video retrieval researchers have begun to create various representations or surrogates for digital videos, such as poster frames, storyboards, video skims and fast forwards. How to evaluate the effectiveness of these video surrogates has become an issue for researchers. This paper proposes two general classes of user tasks—recognition tasks and tasks requiring inference—for which performance measures were developed. The measures include graphical object recognition, textual object recognition, action recognition, free-text gist determination, multiple-choice gist determination and visual gist determination. The preliminary results from two user studies applying these six measures are also discussed in this paper.

This paper describes a study on the image searching behavior of end-users (journalists) and intermediaries (archivists) in a newspaper editorial office. Image queries by end-users and requests to intermediaries were analyzed, compared and categorized according to typologies from literature. The process of image selection was modeled and selection criteria were studied based on interviews, observation and a survey. The results indicate that most image queries and requests dealt with specific entities, but that object types were also common. Thematic image needs seem to be fulfilled by end-user searching and browsing rather than by requests. Image retrieval tasks were highly influenced by contextual factors. Relevance assessments were made at situational level using several types of criteria, including abstract and affective factors. Several types of collaborative searches were observed. Richer research and analysis methods are needed to characterize journalists ' image needs and searching behavior.

The Open Video Project is specifically concerned with the surrogates that can represent the objects in a digital video collection and the mechanisms through which people can manipulate those surrogates. In TREC VID 2003, we compared the effectiveness of a transcript-only search system, a features-only search system and a search system combining transcript and feature searching. We also presented several different views for users to browse the results pages: a horizontal view, a vertical view, a “before & after ” view, and an extra-keyframe view. A within-subjects research design was used, so that each of the 36 participants was exposed to all three search systems. Each participant searched half (12) of the assigned topics. The user satisfaction measures recommended by NIST were augmented by measurements of participants ’ perceived usefulness, perceived ease of use, and flow. Results indicated that, with the transcript-only system and the combined system, users were able to achieve higher recall in less time per search. The results from the measures of satisfaction indicate that the users found the transcript-only and combined systems to be more useful and easier to use, and their use resulted in stronger perceptions of enjoyment and concentration than the features-only system. It is concluded that, as users gain experience with features searching, it will be a welcome supplement to transcript searching. 1

Some documents provoke emotions in people viewing them. Will it be possible to describe emotions consistently and use this information in retrieval systems? We tested collective (statistically aggregated) emotion indexing using images as examples. Considering psychological results, basic emotions are anger, disgust, fear, happiness, and sadness. This study follows an approach developed by Lee and Neal (2007) for music emotion retrieval and applies scroll bars for tagging basic emotions and their intensities. A sample comprising 763 persons tagged emotions caused by images (retrieved from www.Flickr.com) applying scroll bars and (linguistic) tags. Using SPSS, we performed descriptive statistics and correlation analysis. For more than half of the images, the test persons have clear emotion favorites. There are prototypical images for given emotions. The document-specific consistency of tagging using a scroll bar is, for some images, very high. Most of the (most commonly used) linguistic tags are on the basic level (in the sense of Rosch’s basic level theory). The distributions of the linguistic tags in our examples follow an inverse power-law. Hence, it seems possible to apply collective image emotion tagging to image information systems and to present a new search option for basic emotions. This article is one of the first steps in the research area of emotional information retrieval (EmIR).