Interpolating between types and tokens by estimating power-law generators

Interpolating between types and tokens by estimating power-law generators (2006)

Cached

Download Links

Other Repositories/Bibliography

DBLP

by Sharon Goldwater , Thomas L. Griffiths , Mark Johnson

Venue:	In Advances in Neural Information Processing Systems 18
Citations:	56 - 12 self

Versions

Version History

Metadata Version 1

Datum	Value	Source
TITLE	Interpolating between types and tokens by estimating power-law generators	INFERENCE
AUTHOR NAME	Sharon Goldwater	SVM HeaderParse 0.2
AUTHOR AFFIL	Department of Cognitive and Linguistic Sciences; Brown University, Providence RI 02912	SVM HeaderParse 0.2
AUTHOR ADDR	USA	SVM HeaderParse 0.2
AUTHOR NAME	Thomas L. Griffiths	SVM HeaderParse 0.2
AUTHOR AFFIL	Department of Cognitive and Linguistic Sciences; Brown University, Providence RI 02912	SVM HeaderParse 0.2
AUTHOR ADDR	USA	SVM HeaderParse 0.2
AUTHOR NAME	Mark Johnson	SVM HeaderParse 0.2
AUTHOR AFFIL	Department of Cognitive and Linguistic Sciences; Brown University, Providence RI 02912	SVM HeaderParse 0.2
AUTHOR ADDR	USA	SVM HeaderParse 0.2
ABSTRACT	Standard statistical models of language fail to capture one of the most striking properties of natural languages: the power-law distribution in the frequencies of word tokens. We present a framework for developing statistical models that generically produce power-laws, augmenting standard generative models with an adaptor that produces the appropriate pattern of token frequencies. We show that taking a particular stochastic process – the Pitman-Yor process – as an adaptor justifies the appearance of type frequencies in formal analyses of natural language, and improves the performance of a model for unsupervised learning of morphology. 1	SVM HeaderParse 0.2
YEAR	2006	INFERENCE
VENUE	In Advances in Neural Information Processing Systems 18	INFERENCE
VENUE TYPE	CONFERENCE	INFERENCE
PAGES	18	INFERENCE
CITATIONS	15 found	ParsCit 1.0