Exploring web scale language models for search query processing

Exploring web scale language models for search query processing

Cached

Download Links

by Jian Huang , Xiaolong Li , Jianfeng Gao , Kuansan Wang , Jiangbo Miao , Fritz Behr

Venue:	In Proceedings of WWW 2010
Citations:	11 - 7 self

Versions

Version History

Metadata Version 1

Datum	Value	Source
TITLE	Exploring web scale language models for search query processing	INFERENCE
AUTHOR NAME	Jian Huang	SVM HeaderParse 0.2
AUTHOR AFFIL	Information Sciences and; Technology; Pennsylvania State University	SVM HeaderParse 0.2
AUTHOR ADDR	University Park, PA 16802	SVM HeaderParse 0.2
AUTHOR NAME	Xiaolong Li	SVM HeaderParse 0.2
AUTHOR AFFIL	Microsoft Research; One Microsoft	SVM HeaderParse 0.2
AUTHOR ADDR	Way; Redmond, WA 98052	SVM HeaderParse 0.2
AUTHOR NAME	Jianfeng Gao	SVM HeaderParse 0.2
AUTHOR AFFIL	Microsoft Research; One Microsoft	SVM HeaderParse 0.2
AUTHOR ADDR	Way; Redmond, WA 98052	SVM HeaderParse 0.2
AUTHOR NAME	Kuansan Wang	SVM HeaderParse 0.2
AUTHOR AFFIL	Microsoft Research; One Microsoft	SVM HeaderParse 0.2
AUTHOR ADDR	Way; Redmond, WA 98052	SVM HeaderParse 0.2
AUTHOR NAME	Jiangbo Miao	SVM HeaderParse 0.2
AUTHOR AFFIL	Facebook	SVM HeaderParse 0.2
AUTHOR ADDR	704 Hibiscus Place; San Jose, CA, 95117	SVM HeaderParse 0.2
AUTHOR NAME	Fritz Behr	SVM HeaderParse 0.2
AUTHOR AFFIL	Microsoft Corporation; One Microsoft	SVM HeaderParse 0.2
AUTHOR ADDR	Way; Redmond, WA 98052	SVM HeaderParse 0.2
ABSTRACT	It has been widely observed that search queries are composed in a very different style from that of the body or the title of a document. Many techniques explicitly accounting for this language style discrepancy have shown promising results for information retrieval, yet a large scale analysis on the extent of the language differences has been lacking. In this paper, we present an extensive study on this issue by examining the language model properties of search queries and the three text streams associated with each web document: the body, the title, and the anchor text. Our information theoretical analysis shows that queries seem to be composed in a way most similar to how authors summarize documents in anchor texts or titles, offering a quantitative explanation to the observations in past work. We apply these web scale n-gram language models to three search query processing (SQP) tasks: query spelling correction, query bracketing and long query segmentation. By controlling the size and the order of different language models, we find that the perplexity metric to be a good accuracy indicator for these query processing tasks. We show that using smoothed language models yields significant accuracy gains for query bracketing for instance, compared to using web counts as in the literature. We also demonstrate that applying web-scale language models can have marked accuracy advantage over smaller ones.	SVM HeaderParse 0.2
VENUE	In Proceedings of WWW 2010	INFERENCE
VENUE TYPE	CONFERENCE	INFERENCE
PAGES	451--460	INFERENCE
CITATIONS	30 found	ParsCit 1.0