Results 11 - 20
of
40
Robust Ranking Models via Risk-Sensitive Optimization
"... Many techniques for improving search result quality have been proposed. Typically, these techniques increase average effectiveness by devising advanced ranking features and/or by developing sophisticated learning to rank algorithms. However, while these approaches typically improve average performan ..."
Abstract
-
Cited by 7 (0 self)
- Add to MetaCart
(Show Context)
Many techniques for improving search result quality have been proposed. Typically, these techniques increase average effectiveness by devising advanced ranking features and/or by developing sophisticated learning to rank algorithms. However, while these approaches typically improve average performance of search results relative to simple baselines, they often ignore the important issue of robustness. That is, although achieving an average gain overall, the new models often hurt performance on many queries. This limits their application in real-world retrieval scenarios. Given that robustness is an important measure that can negatively impact user satisfaction, we present a unified framework for jointly optimizing effectiveness and robustness. We propose an objective that captures the tradeoff between these two competing measures and demonstrate how we can jointly optimize for these two measures in a principled learning framework. Experiments indicate that ranking models learned this way significantly decreased the worst ranking failures while maintaining strong average effectiveness on par with current state-of-the-art models.
Characterizing and Supporting Cross-Device Search Tasks
"... fine dining in seattle, wa italian restaurants in seattle restaurants barolo menu 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 Web searchers frequently transition from desktop computers and laptops to mobile devices, and vice versa. Little is known about the nature of cross-devic ..."
Abstract
-
Cited by 6 (2 self)
- Add to MetaCart
(Show Context)
fine dining in seattle, wa italian restaurants in seattle restaurants barolo menu 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 Web searchers frequently transition from desktop computers and laptops to mobile devices, and vice versa. Little is known about the nature of cross-device search tasks, yet they represent an important opportunity for search engines to help their users, especially those on the target (post-switch) device. For example, the search engine could save the current session and re-instate it post switch, or it could capitalize on down-time between devices to proactively retrieve content on behalf of the searcher. In this paper, we present a log-based study to define and characterize cross-device search behavior and predict the resumption of cross-device tasks. Using data from a large commercial search engine, we show that there are discernible and noteworthy patterns of search behavior associated with device transitions. We also develop learned models for predicting task resumption on the target device using behavioral, topical, geospatial, and temporal features. Our findings show that our models can attain strong prediction accuracy and have direct implications for the development of tools to help people search more effectively in a multi-device world.
Beliefs and Biases in Web Search
"... People’s beliefs, and unconscious biases that arise from those beliefs, influence their judgment, decision making, and actions, as is commonly accepted among psychologists. Biases can be observed in information retrieval in situations where searchers seek or are presented with information that signi ..."
Abstract
-
Cited by 3 (2 self)
- Add to MetaCart
(Show Context)
People’s beliefs, and unconscious biases that arise from those beliefs, influence their judgment, decision making, and actions, as is commonly accepted among psychologists. Biases can be observed in information retrieval in situations where searchers seek or are presented with information that significantly deviates from the truth. There is little understanding of the impact of such biases in search. In this paper we study search-related biases via multiple probes: an exploratory retrospective survey, human labeling of the captions and results returned by a Web search engine, and a largescale log analysis of search behavior on that engine. Targeting yesno questions in the critical domain of health search, we show that Web searchers exhibit their own biases and are also subject to bias from the search engine. We clearly observe searchers favoring positive information over negative and more than expected given base rates based on consensus answers from physicians. We also show that search engines strongly favor a particular, usually positive, perspective, irrespective of the truth. Importantly, we show that these biases can be counterproductive and affect search outcomes; in our study, around half of the answers that searchers settled on were actually incorrect. Our findings have implications for search engine design, including the development of ranking algorithms that consider the desire to satisfy searchers (by validating their beliefs) and providing accurate answers and properly considering base rates. Incorporating likelihood information into search is particularly important for consequential tasks, such as those with a medical focus.
Identifying users’ topical tasks in web search,”
- in Proceedings of the Sixth ACM International Conference on Web Search and Data Mining, ser. WSDM ’13.
, 2013
"... ABSTRACT A search task represents an atomic information need of a user in web search. Tasks consist of queries and their reformulations, and identifying tasks is important for search engines since they provide valuable information for determining user satisfaction with search results, predicting us ..."
Abstract
-
Cited by 3 (2 self)
- Add to MetaCart
(Show Context)
ABSTRACT A search task represents an atomic information need of a user in web search. Tasks consist of queries and their reformulations, and identifying tasks is important for search engines since they provide valuable information for determining user satisfaction with search results, predicting user search intent, and suggesting queries to the user. Traditional approaches to identifying tasks exploit either temporal or lexical features of queries. However, many query refinements are topical, which means that a query and its refinements may not be similar on the lexical level. Furthermore, multiple tasks in the same search session may interleave, which means we cannot simply order the searches by their timestamps and divide the session into multiple tasks. Thus, in order to identify tasks correctly, we need to be able to compare two queries at the semantic level. In this paper, we use a knowledgebase known as Probase to infer the conceptual meanings of queries, and automatically identify the topical query refinements in the tasks. Experimental results on real search log data demonstrate that Probase can indeed help estimate the topical affinity between queries, and thus enable us to merge queries that are topically related but dissimilar at the lexical level.
Supporting Complex Search Tasks
"... We present methods to automatically identify and recommend sub-tasks to help people explore and accomplish complex search tasks. Although Web searchers often exhibit directed search behaviors such as navigating to a particular Website or locating a particular item of information, many search scenari ..."
Abstract
-
Cited by 3 (0 self)
- Add to MetaCart
(Show Context)
We present methods to automatically identify and recommend sub-tasks to help people explore and accomplish complex search tasks. Although Web searchers often exhibit directed search behaviors such as navigating to a particular Website or locating a particular item of information, many search scenarios involve more complex tasks such as learning about a new topic or planning a vacation. These tasks often involve multiple search queries and can span mul-tiple sessions. Current search systems do not provide adequate sup-port for tackling these tasks. Instead, they place most of the burden on the searcher for discovering which aspects of the task they should explore. Particularly challenging is the case when a searcher lacks the task knowledge necessary to decide which step to tackle next. In this paper, we propose methods to automatically mine search logs for tasks and build an association graph connecting multiple tasks together. We then leverage the task graph to assist new searchers in exploring new search topics or tackling multi-step search tasks. We demonstrate through experiments with human participants that we can discover related and interesting tasks to as-sist with complex search scenarios.
Understanding How People Interact with Web Search Results that Change in Real-Time using Implicit Feedback
- ACM International Conference on Information and Knowledge Management
"... ABSTRACT The way a searcher interacts with query results can reveal a lot about what is being sought. Considerable research has gone into using implicit relevance feedback to identify relevant content in real-time, but little is known about how to best present this newly identified relevant content ..."
Abstract
-
Cited by 3 (1 self)
- Add to MetaCart
(Show Context)
ABSTRACT The way a searcher interacts with query results can reveal a lot about what is being sought. Considerable research has gone into using implicit relevance feedback to identify relevant content in real-time, but little is known about how to best present this newly identified relevant content to users. In this paper we compare a traditional search interface with one that dynamically re-ranks and recommends search results as the user interacts with it in order to build a picture of how and when users should be offered dynamically identified relevant content. We present several studies that compare logged behavior for hundreds of thousands of users and millions of queries as well as self-reported measures of success across the two interaction models. Compared to traditional web search, users presented with dynamically ranked results exhibit higher engagement and find information faster, particularly during exploratory tasks. These findings have implications for how search engines might best exploit implicit feedback in realtime in order to help users identify the most relevant results as quickly as possible. Categories and Subject Descriptors Keywords Interactive information retrieval, query log analysis, web search, dynamic ranked retrieval, implicit relevance feedback.
From Devices to People: Attribution of Search Activity in Multi-User Settings
"... Online services rely on unique identifiers of machines to tailor of-ferings to their users. An implicit assumption is made that each ma-chine identifier maps to an individual. However, shared machines are common, leading to interwoven search histories and noisy sig-nals for applications such as pers ..."
Abstract
-
Cited by 3 (2 self)
- Add to MetaCart
(Show Context)
Online services rely on unique identifiers of machines to tailor of-ferings to their users. An implicit assumption is made that each ma-chine identifier maps to an individual. However, shared machines are common, leading to interwoven search histories and noisy sig-nals for applications such as personalized search and advertising. We present methods for attributing search activity to individual searchers. Using ground truth data for a sample of almost four mil-lion U.S. Web searchers—containing both machine identifiers and person identifiers—we show that over half of the machine identifi-ers comprise the queries of multiple people. We characterize varia-tions in features of topic, time, and other aspects such as the com-plexity of the information sought per the number of searchers on a machine, and show significant differences in all measures. Based on these insights, we develop models to accurately estimate when multiple people contribute to the logs ascribed to a single machine identifier. We also develop models to cluster search behavior on a machine, allowing us to attribute historical data accurately and au-tomatically assign new search activity to the correct searcher. The findings have implications for the design of applications such as personalized search and advertising that rely heavily on machine identifiers to custom-tailor their services.
Text selections as implicit relevance feedback
- SIGIR
, 2012
"... Users ’ search activity has been used as implicit feedback to model search interests and improve the performance of search systems. In search engines, this behavior usually takes the form of queries and result clicks. However, richer data on how people engage with search results can now be captured ..."
Abstract
-
Cited by 2 (0 self)
- Add to MetaCart
(Show Context)
Users ’ search activity has been used as implicit feedback to model search interests and improve the performance of search systems. In search engines, this behavior usually takes the form of queries and result clicks. However, richer data on how people engage with search results can now be captured at scale, creating new opportunities to enhance search. In this poster we focus on one type of newly-observable behavior: text selection events on search-result captions. We show that we can use text selections as implicit feedback to significantly improve search result relevance.
A Noise-aware Click Model for Web Search
"... Recent advances in click model have established it as an attractive approach to infer document relevance. Most of these advances consider the user click/skip behavior as binary events but neglect the context in which a click happens. We show that real click behavior in industrial search engines is o ..."
Abstract
-
Cited by 2 (1 self)
- Add to MetaCart
(Show Context)
Recent advances in click model have established it as an attractive approach to infer document relevance. Most of these advances consider the user click/skip behavior as binary events but neglect the context in which a click happens. We show that real click behavior in industrial search engines is often noisy and not always a good indication of relevance. For a considerable percentage of clicks, users select what turn out to be irrelevant documents and these clicks should not be directly used as evidence for relevance inference. Thus in this paper, we put forward an observation that the relevance indication degree of a click is not a constant, but can be differentiated by user preferences and the context in which the user makes her click decision. In particular, to interpret the click behavior discriminatingly, we propose a Noise-aware Click Model (NCM) by characterizing the noise degree of a click, which indicates the quality of the click for inferring relevance. Specifically, the lower the click noise is, the more important the click is in its role for relevance inference. To verify the necessity of explicitly accounting for the uninformative noise in a user click, we conducted experiments on a billion-scale dataset. Extensive experimental results demonstrate that as compared with two state-of-theart click models in Web Search, NCM can better interpret user click behavior and achieve significant improvements in terms of both perplexity and NDCG.
Cohort Modeling for Enhanced Personalized Search
"... Web search engines utilize behavioral signals to develop search ex-periences tailored to individual users. To be effective, such person-alization relies on access to sufficient information about each user’s interests and intentions. For new users or new queries, profile in-formation may be sparse or ..."
Abstract
-
Cited by 2 (2 self)
- Add to MetaCart
(Show Context)
Web search engines utilize behavioral signals to develop search ex-periences tailored to individual users. To be effective, such person-alization relies on access to sufficient information about each user’s interests and intentions. For new users or new queries, profile in-formation may be sparse or non-existent. To handle these cases, and perhaps also improve personalization for those with profiles, search engines can employ signals from users who are similar along one or more dimensions, i.e., those in the same cohort. In this paper we describe a characterization and evaluation of the use of such cohort modeling to enhance search personalization. We experiment with three pre-defined cohorts—topic, location, and top-level domain preference—independently and in combination, and also evaluate methods to learn cohorts dynamically. We show via extensive ex-perimentation with large-scale logs from a commercial search en-gine that leveraging cohort behavior can yield significant relevance gains when combined with a production search engine ranking al-gorithm that uses similar classes of personalization signal but at the individual searcher level. Additional experiments show that our gains can be extended when we dynamically learn cohorts and tar-get easily-identifiable classes of ambiguous or unseen queries.
