In this paper we describe ImageCLEF 1, the cross language image retrieval track of the Cross Language Evaluation Forum (CLEF 3). We instigated and ran a pilot experiment in 2003 where participants submitted entries for an ad hoc bilingual image retrieval task on a collection of historic photographs from St. Andrews University Library. This was designed to simulate the situation in which users would express their search request in natural language but require visual documents in return. For 2004 we have extended the tasks to include a medical image retrieval task and a user-centred evaluation.

We present a comprehensive strategy for evaluating image retrieval algorithms. Because automated image retrieval is only meaningful in its service to people, performance characterization must be grounded in human evaluation. Thus we have collected a large data set of human evaluations of retrieval results, both for query by image example and query by text. The data is independent of any particular image retrieval algorithm and can be used to evaluate and compare many such algorithms without further data collection. The data and calibration software are available on-line

Abstract. In this paper we describe our proposal for a cross language image retrieval task called ImageCLEF 1 being run as part of the Cross Language Evaluation Forum (CLEF 2). A pilot experiment was organised for 2003 in which participants performed an ad hoc bilingual image retrieval task on a collection of historic photographs to simulate the situation in which users express their search request in natural language but require visual documents in return. For 2004 we plan to extend the tasks to include a medical image retrieval task and user-centred evaluation. 1

Abstract. Semantic classification of images using low-level features is a challenging problem. Combining experts with different classifier structures, trained by MPEG-7 low-level color and texture descriptors is examined as a solution alternative. For combining different classifiers and features, two advanced decision mechanisms are proposed, one of which enjoys a significant classification performance improvement. Simulations are conducted on 8 different visual semantic classes, resulting in accuracy improvements between 3.5-6.5%, when they are compared with the best performance of single classifier systems. 1

We address evaluation of image understanding and retrieval large scale image data in the context of three evaluation projects. The first project is a comprehensive strategy for evaluating image retrieval algorithms and provides an open reference data set for doing so. The second project develops word prediction as a semantically relevant evaluation strategy, and applies it to the evaluation of of image processing methods for semantic image analysis. The third project evaluates words for suitability of their visual properties for use in an image annotation framework.

This thesis aims to augment the Geographic Information Retrieval process with information extracted from world knowledge. This aim is approached from three directions: classifying world knowledge, disambiguating placenames and modelling users. Geographic information is becoming ubiquitous across the Internet, with a significant proportion of web documents and web searches containing geographic entities, and the proliferation of Internet enabled mobile devices. Traditional information retrieval treats these geographic entities in the same way as any other textual data. In this thesis I augment the retrieval process with geographic information, and show how methods built upon world knowledge outperform methods based on heuristic rules. The source of world knowledge used in this thesis is Wikipedia. Wikipedia has become a phenomenon of the Internet age and needs little introduction. As a linked corpus of semi-structured data, it is unsurpassed. Two approaches to mining information from Wikipedia are rigorously explored: initially I classify Wikipedia articles into broad categories; this is followed by much finer classification where Wikipedia articles are disambiguated as specific locations. The thesis concludes with the proposal of the Steinberg hypothesis: By analysing a range of wikipedias in different languages I demonstrate that a