We describe results from Web search log studies aimed at elucidating user behaviors associated with queries and destination URLs that appear with different frequencies. We note the diversity of information goals that searchers have and the differing ways that goals are specified. We examine rare and common information goals that are specified using rare or common queries. We identify several significant differences in user behavior depending on the rarity of the query and the destination URL. We find that searchers are more likely to be successful when the frequencies of the query and destination URL are similar. We also establish that the behavioral differences observed for queries and goals of varying rarity persist even after accounting for potential confounding variables, including query length, search engine ranking, session duration, and task difficulty. Finally, using an information-theoretic measure of search difficulty, we show that the benefits obtained by search and navigation actions depend on the frequency of the information goal.

The use of data stored in transaction logs of Web search engines, Intranets, and Web sites can provide valuable insight into understanding the information-searching process of online searchers. This understanding can enlighten information system design, interface development, and devising the information architecture for content collections. This article presents a review and foundation for conducting Web search transaction log analysis. A methodology is outlined consisting of three stages, which are collection, preparation, and analysis. The three stages of the methodology are presented in detail with discussions of goals, metrics, and processes at each stage. Critical terms in transaction log analysis for Web searching are defined. The strengths and limitations of transaction log analysis as a research method are presented. An application to log client-side interactions that supplements transaction logs is reported on, and the application is made available for use by the research community. Suggestions are provided on ways to leverage the strengths of, while addressing the limitations of, transaction log analysis for Web-searching research. Finally, a complete flat text transaction log from a commercial search engine is available as supplementary material with this

In this article, we report results of an investigation into the effect of sponsored links on ecommerce information seeking on the Web. In this research, 56 participants each engaged in six ecommerce Web searching tasks. We extracted these tasks from the transaction log of a Web search engine, so they represent actual ecommerce searching information needs. Using 60 organic and 30 sponsored Web links, the quality of the Web search engine results was controlled by switching nonsponsored and sponsored links on half of the tasks for each participant. This allowed for investigating the bias toward sponsored links while controlling for quality of content. The study also investigated the relationship between searching self-efficacy, searching experience, types of ecommerce information needs, and the order of links on the viewing of sponsored links. Data included 2,453 interactions with links from result pages and 961 utterances evaluating these links. The results of the study indicate that there is a strong preference for nonsponsored links, with searchers viewing these results first more than 82 % of the time. Searching self-efficacy and experience does not increase the likelihood of viewing sponsored links, and the order of the result listing does not appear to affect searcher evaluation of sponsored links. The implications for sponsored links as a longterm business model are discussed.

Metasearch engines are an intuitive method for improving the performance of Web search by increasing coverage, returning large numbers of results with a focus on relevance, and presenting alternative views of information needs. However, the use of metasearch engines in an operational environment is not well understood. In this study, we investigate the usage of Dogpile.com, a major Web metasearch engine, with the aim of discovering how Web searchers interact with metasearch engines. We report results examining 2,465,145 interactions from 534,507 users of Dogpile.com on May 6, 2005 and compare these results with findings from other Web searching studies. We collect data on geographical location of searchers, use of system feedback, content selection, sessions, queries, and term usage. Findings show that Dogpile.com searchers are mainly from the USA (84 % of searchers), use about 3 terms per query (mean � 2.85), implement system feedback moderately (8.4 % of users), and generally (56 % of users) spend less than one minute interacting with the Web search engine. Overall, metasearchers seem to have higher degrees of interaction than searchers on non-metasearch engines, but their sessions are for a shorter period of time. These aspects of metasearching may be what define the differences from other forms of Web searching. We discuss the implications of our findings in relation to metasearch for Web searchers, search engines, and content providers.

In this paper, we report results of an investigation into the effect of sponsored links on ecommerce information seeking on the Web. In this research, 56 participants each engaged in 6 ecommerce Web searching tasks using 60 organic and 30 sponsored Web links for each task. We extracted these tasks from the transaction log of an actual Web search engine, so these queries represent actual ecommerce searching information needs. In the study, we controlled for quality of the Web search engine results by switching organic and sponsored links on three of the six searching tasks for each participant. We counterbalanced the order of presentation among participants. We investigated the perceptions of sponsored links and the factors that influence this bias. Data included 2,453 interactions with links from result pages and 961 utterances evaluating these links. Findings include that there is a strong preference for organic links, a bias against sponsored results, and that more than 56 % of the time, the title of the sponsored link was the determining factor in searcher perceived relevance. We discuss implications for sponsored links and paid search as a long-term business model.

Domain experts search differently than people with little or no domain knowledge. Previous research suggests that domain experts employ different search strategies and are more successful in finding what they are looking for than non-experts. In this paper we present a large-scale, longitudinal, log-based analysis of the effect of domain expertise on web search behavior in four different domains (medicine, finance, law, and computer science). We characterize the nature of the queries, search sessions, web sites visited, and search success for users identified as experts and non-experts within these domains. Large-scale analysis of real-world interactions allows us to understand how expertise relates to vocabulary, resource use, and search task under more realistic search conditions than has been possible in previous small-scale studies. Building upon our analysis we develop a model to predict expertise based on search behavior, and describe how knowledge about domain expertise can be used to present better results and query suggestions to users and to help non-experts gain expertise.

The practice of guiding a search engine based on query logs observed from the engine's user population provides large volumes of data but potentially also sacrifices the privacy of the user. In this paper, we ask the following question: Is it possible, given rich instrumented data from a panel and usability study data, to observe complete information without routinely analyzing query logs? What unique benefits to the user could hypothetically be derived from analyzing query logs? We demonstrate that three different modes of collecting data, the field study, the instrumented user panel, and the raw query log, provide complementary sources of data. The query log is the least rich source of data for individual events, but has irreplaceable information for understanding the scope of resources that a search engine needs to provide for the user.

Accurate topical classification of user queries allows for increased effectiveness and efficiency in general-purpose Web search systems. Such classification becomes critical if the system must route queries to a subset of topic-specific and resource-constrained back-end databases. Successful query classification poses a challenging problem, as Web queries are short, thus providing few features. This feature sparseness, coupled with the constantly changing distribution and vocabulary of queries, hinders traditional text classification. We attack this problem by combining multiple classifiers, including exact lookup and partial matching in databases of manually classified frequent queries, linear models trained by supervised learning, and a novel approach based on mining selectional preferences from a large unlabeled query log. Our approach classifies queries without using external sources of information, such as online Web directories or the contents of retrieved pages, making it viable for use in demanding operational environments, such as large-scale Web search services. We evaluate our approach using a large sample of queries from an operational Web search engine and show that our combined method increases recall by nearly 40 % over the best single method while maintaining adequate precision. Additionally, we compare our results to those from the 2005 KDD Cup and find that we perform competitively despite our operational restrictions. This suggests it is possible to topically classify a significant portion of the query stream without requiring external sources of information, allowing for deployment in operationally restricted environments.

Analysis of data from a major metasearch engine reveals that sponsored-link click-through rates appear lower than previously reported. Combining sponsored and nonsponsored links in a single listing, while providing some benefits to users, does not appear to increase clicks on sponsored listings. Search engines have become indispensable to interacting on the Web. In addition to processing information requests, they are navigational tools that can direct users to specific Web sites or aid in browsing. Search engines can also facilitate e-commerce transactions as well as provide access to noncommercial services such as maps, online auctions, and driving directions. People use search engines as dictionaries, spell checkers, and thesauruses; as discussion groups (Google Groups) and social networking forums (Yahoo! Answers); and even as entertainment

Current Web search tools do a good job of retrieving documents that satisfy the most common intentions associated with a query, but do not do a very good job of discerning different individuals ’ unique search goals. We explore the variation in what different people consider relevant to the same query by mining three data sources: 1) explicit relevance judgments, 2) clicks on search results (a behavior-based implicit measure of relevance), and 3) the similarity of desktop content to search results (a content-based implicit measure of relevance). We find that people’s explicit judgments for the same queries differ greatly. As a result, there is a large gap between how well search engines could perform if they were to tailor results to the individual, and how well they currently perform by returning results designed to satisfy everyone. We call this gap the potential for personalization. The two implicit indicators we studied provide complementary value for approximating this variation in result relevance among people. We discuss several uses of our findings, including a personalized search system that takes advantage of the implicit measures by ranking personally relevant results more highly and improving click-through rates.