An adaptive recommendation service seeks to adapt to its users, providing increasingly personalized recommendations over time. In this paper we introduce the "Fab" adaptive web page recommendation service. There has been much research on analyzing document content in order to improve recommendations or search results. More recently researchers have begun to explore how the similarities between users can be exploited to the same ends. The Fab system strikes a balance between these two approaches, taking advantage of the shared interests among users without losing the benefits of the representations provided by content analysis. Running since March 1996, it has been populated with a collection of agents for the collection and selection of web pages, whose interaction fosters emergent collaborative properties. In this paper we explain the design of the system architecture and report the results of our first experiment, evaluating recommendations provided to a group of test users. 1 Introd...

We investigated the sources and effectiveness of search actions are still evolving, changing, and, at times, improvterms used during mediated on-line searching under real-life (as opposed to laboratory) circumstances. A stratified model of information retrieval (IR) interaction served as a framework for the analysis. For the analysis, we used the on-line transaction logs, videotapes, and ing. However, we still do not fully understand the many complex aspects of interactive processes, despite a num-ber of theoretical and experimental studies and scholarly treatises (Ingwerson, 1992, 1996). Furthermore, most of transcribed dialogue of the presearch and on-line interaction between 40 users and 4 professional intermediaries. Each user provided one question and interacted with one of the four intermediaries. Searching was done using DIALOG. Five sources of search terms were identified:

this paper, we show that evaluation methods developed for standard test collection must be modified in order to deal with retrieval of structured documents. Specifically, size and overlap of document components must be taken into account. For this purpose, we use coverage in addition to relevance as evaluation criterion, in combination with multi-valued scales for both. A new quality metric based on the notion of concept spaces is developed. We compare the results of this new metric with the results obtained by the metric used in the first round of INEX, in 2002

A new laboratory-based method for the evaluation of Boolean queries in free-text searching of full-text databases is proposed. The method is based on a controlled formulation of inclusive query plans, on an automatic conversion of query plans into a set of elementary queries, and on composing optimal queries at varying operational levels by combining appropriate sub-sets of elementary queries. The method is based on the idea of reverse engineering, and exploits full relevance data of documents to find the query performing optimally within given operational constraints. The proposed

A growing body of studies is developing approaches to evaluating human interaction with Web search engines, including the usability and effectiveness of Web search tools. This study explores a user-centered approach to the evaluation of the Web search engine Inquirus – a Web meta-search tool developed by researchers from the NEC Research Institute. The goal of the study reported in this paper was to develop a user-centered approach to the evaluation including: (1) effectiveness: based on the impact of users ' interactions on their information problem and information seeking stage, and (2) usability: including screen layout and system capabilities for users. Twenty-two (22) volunteers searched Inquirus on their own personal information topics. Data analyzed included: (1) user pre- and post-search questionnaires and (2) Inquirus search transaction logs. Key findings include: (1) Inquirus was rated highly by users on various usability measures, (2) all users experienced some level of shift/change in their information problem, information seeking, and personal knowledge due to their Inquirus interaction, (3) different users experienced different levels of change/shift, and (4) the search measure precision did not correlate with other user-based measures. Some users experienced major changes/shifts in various user-based variables, such as information problem or information seeking stage with a search of low precision and vice versa. Implications for the development of user-centered approaches to the evaluation of Web and IR systems and further research are discussed. 2

Information Retrieval (IR) is the discipline that deals with retrieval of unstructured data, especially textual documents, in response to a query or topic statement, which may itself be unstructured, e.g., a sentence or even another document, or which may be structured, e.g., a boolean expression. The need for effective methods of automated IR has grown in importance because of the tremendous explosion in the amount of unstructured data, both internal, corporate document collections, and the immense and growing number of document sources on the Internet. This report is a tutorial and survey of the state of the art, both research and commercial, in this dynamic field. The topics covered include: formulation of structured and unstructured queries and topic statements, indexing (including term weighting) of document collections, methods for computing the similarity of queries and documents, classification and routing of documents in an incoming stream to users on the basis of topic or nee...

Information retrieval on the Web is very different from retrieval in traditional indexed databases. This difference arises from: the high degree of dynamism of the Web; its hyper-linked character; the absence of a controlled indexing vocabulary; the heterogeneity of document types and authoring styles; the easy access that different types of users may have to it. Thus, since Web retrieval is substantially different from information retrieval, new or revised evaluative measures are required to assess retrieval performance using Web search engines. This paper suggests a number of different measures to evaluate information retrieval from the Web. The motivation behind each of these measures is presented, along with their descriptions and definitions. In the second part of the paper, application of these measures is illustrated in the evaluation of three search engines. The purpose of this paper is not to give the definite prescription for evaluating information retrieval from the Web, but rather to present some examples and to initiate a wider discussion of how to enhance measures of Web search performance.

Peer network systems are becoming an increasingly important development in Web search technology. Many studies show that peer search systems perform better when a query is sent to a group of peers semantically similar to the query. This suggests that semantic communities should form so that a query can quickly propagate to many appropriate peers. For the network to be functional, its dynamic communication topology must match the semantic clustering of peers. We introduce two criteria to evaluate a peer search network based on the concept of semantic locality: first, the “smallworld” topology of the network; second, we use topical semantic similarity to monitor the quality of a peer’s neighbors over time by looking at whether a peer chooses semantically appropriate neighbors to route its queries. We present several simulation experiments conducted with different peer search algorithms on our peer Web search system, 6S. The results suggest that 6S, despite its use of an unstructured overlay network; can effectively foster the spontaneous formation of semantic communities through local peer interactions alone.

Performance evaluation is an important issue in Web search engine researches. Traditional evaluation methods rely on much human efforts and are therefore quite time-consuming. With clickthrough data analysis, we proposed an automatic search engine performance evaluation method. This method generates navigational type query topics and answers automatically based on search users ’ querying and clicking behavior. Experimental results based on a commercial Chinese search engine’s user logs show that the automatically method gets a similar evaluation result with traditional assessor-based ones.

This paper addresses the problem of accessing very large heterogeneous document collections by proposing a new approach to using clustering for information retrieval: mediated access through a clustered collection. In what is actually an information access environment, the user can explore a relatively small, well structured, pre-clustered collection covering a particular subject domain, in order to understand the concepts encompassed and to clarify and refine his/her information need. The user can ostensively indicate clusters and documents of interest and be assisted in formulating a query, based on which a search can be done on a large, non-structured collection. Finally, the original cluster structure is the basis for visualisation tools that allow the user to explore search results. WebCluster, the system implementing these ideas, is presented, together with results of an initial formative experiment and plans for future experiments.