A Stochastic Memoizer for Sequence Data

A Stochastic Memoizer for Sequence Data

Cached

Download Links

by Frank Wood , Cédric Archambeau , Lancelot James , Yee Whye Teh

Citations:

7 - 4 self

Versions

Version 0

Version History

Metadata Version 0

Datum	Value	Source
TITLE	A Stochastic Memoizer for Sequence Data	SVM HeaderParse 0.2
AUTHOR NAME	Frank Wood	SVM HeaderParse 0.2
AUTHOR NAME	Cédric Archambeau	SVM HeaderParse 0.2
AUTHOR NAME	Lancelot James	SVM HeaderParse 0.2
AUTHOR NAME	Yee Whye Teh	SVM HeaderParse 0.2
AUTHOR AFFIL	; ⋆Gatsby Computational Neuroscience Unit; University College London; Centre for Computational Statistics and Machine Learning; University College London; Department of Information and Systems Management	SVM HeaderParse 0.2
AUTHOR ADDR	; 17 Queen Square, London, WC1N 3AR, UK; Gower Street, London, WC1E 6BT, UK; Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong	SVM HeaderParse 0.2
ABSTRACT	We propose an unbounded-depth, hierarchical, Bayesian nonparametric model for discrete sequence data. This model can be estimated from a single training sequence, yet shares statistical strength between subsequent symbol predictive distributions in such a way that predictive performance generalizes well. The model builds on a specific parameterization of an unbounded-depth hierarchical Pitman-Yor process. We introduce analytic marginalization steps (using coagulation operators) to reduce this model to one that can be represented in time and space linear in the length of the training sequence. We show how to perform inference in such a model without truncation approximation and introduce fragmentation operators necessary to do predictive inference. We demonstrate the sequence memoizer by using it as a language model, achieving state-of-the-art results. 1.	SVM HeaderParse 0.2
CITATIONS	16 found	ParsCit 1.0