New Tools for Web-Scale N-grams

New Tools for Web-Scale N-grams

Cached

Download Links

by Dekang Lin , Kenneth Church , Heng Ji , Satoshi Sekine , David Yarowsky , Shane Bergsma , Kailash Patil , Emily Pitler , Rachel Lathbury , Vikram Rao , Kapil Dalwani , Sushant Narsale

Citations:

18 - 10 self

Versions

Version 0

Version History

Metadata Version 0

Datum	Value	Source
TITLE	New Tools for Web-Scale N-grams	SVM HeaderParse 0.2
AUTHOR NAME	Dekang Lin	SVM HeaderParse 0.2
AUTHOR AFFIL	Google, Inc.	SVM HeaderParse 0.2
AUTHOR NAME	Kenneth Church	SVM HeaderParse 0.2
AUTHOR NAME	Heng Ji	SVM HeaderParse 0.2
AUTHOR AFFIL	City University of New York	SVM HeaderParse 0.2
AUTHOR NAME	Satoshi Sekine	SVM HeaderParse 0.2
AUTHOR NAME	David Yarowsky	SVM HeaderParse 0.2
AUTHOR NAME	Shane Bergsma	SVM HeaderParse 0.2
AUTHOR AFFIL	University of Alberta	SVM HeaderParse 0.2
AUTHOR NAME	Kailash Patil	SVM HeaderParse 0.2
AUTHOR NAME	Emily Pitler	SVM HeaderParse 0.2
AUTHOR AFFIL	University of Pennsylvania	SVM HeaderParse 0.2
AUTHOR NAME	Rachel Lathbury	SVM HeaderParse 0.2
AUTHOR AFFIL	University of Virginia	SVM HeaderParse 0.2
AUTHOR NAME	Vikram Rao	SVM HeaderParse 0.2
AUTHOR NAME	Kapil Dalwani	SVM HeaderParse 0.2
AUTHOR NAME	Sushant Narsale	SVM HeaderParse 0.2
AUTHOR AFFIL	University of Virginia	SVM HeaderParse 0.2
ABSTRACT	While the web provides a fantastic linguistic resource, collecting and processing data at web-scale is beyond the reach of most academic laboratories. Previous research has relied on search engines to collect online information, but this is hopelessly inefficient for building large-scale linguistic resources, such as lists of named-entity types or clusters of distributionally-similar words. An alternative to processing web-scale text directly is to use the information provided in an N-gram corpus. An N-gram corpus is an efficient compression of large amounts of text. An N-gram corpus states how often each sequence of words (up to length N) occurs. We propose tools for working with enhanced web-scale N-gram corpora that include richer levels of source annotation, such as part-of-speech tags. We describe a new set of search tools that make use of these tags, and collectively lower the barrier for lexical learning and ambiguity resolution at web-scale. The tools will allow novel sources of information to be applied to long-standing natural language challenges. 1.	SVM HeaderParse 0.2
CITATIONS	32 found	ParsCit 1.0