Results 1 
4 of
4
A hierarchical Bayesian language model based on Pitman–Yor processes
 In Coling/ACL, 2006. 9
, 2006
"... We propose a new hierarchical Bayesian ngram model of natural languages. Our model makes use of a generalization of the commonly used Dirichlet distributions called PitmanYor processes which produce powerlaw distributions more closely resembling those in natural languages. We show that an approxi ..."
Abstract

Cited by 89 (8 self)
 Add to MetaCart
(Show Context)
We propose a new hierarchical Bayesian ngram model of natural languages. Our model makes use of a generalization of the commonly used Dirichlet distributions called PitmanYor processes which produce powerlaw distributions more closely resembling those in natural languages. We show that an approximation to the hierarchical PitmanYor language model recovers the exact formulation of interpolated KneserNey, one of the best smoothing methods for ngram language models. Experiments verify that our model gives cross entropy results superior to interpolated KneserNey and comparable to modified KneserNey. 1
NonParametric Bayesian Areal Linguistics
"... We describe a statistical model over linguistic areas and phylogeny. Our model recovers known areas and identifies a plausible hierarchy of areal features. The use of areas improves genetic reconstruction of languages both qualitatively and quantitatively according to a variety of metrics. We model ..."
Abstract
 Add to MetaCart
(Show Context)
We describe a statistical model over linguistic areas and phylogeny. Our model recovers known areas and identifies a plausible hierarchy of areal features. The use of areas improves genetic reconstruction of languages both qualitatively and quantitatively according to a variety of metrics. We model linguistic areas by a PitmanYor process and linguistic phylogeny by Kingman’s coalescent. 1
On the Pattern Classification of Structured Data using the Neocortexinspired Memoryprediction Framework
, 2009
"... In this master thesis project, we have researched how a theoretical model of the neocortex can be implemented as a hierarchical Bayesian network. The report is based on the theoretical Memoryprediction Framework (MPF) by Hawkins & Blakeslee (2004), which was later implemented in the Hierarchic ..."
Abstract
 Add to MetaCart
(Show Context)
In this master thesis project, we have researched how a theoretical model of the neocortex can be implemented as a hierarchical Bayesian network. The report is based on the theoretical Memoryprediction Framework (MPF) by Hawkins & Blakeslee (2004), which was later implemented in the Hierarchical Temporal Memory (HTM) by George & Hawkins (2005). The assumption of the master thesis project is that the HTM is unable to implement fundamental concepts of the MPF and is furthermore based on methods and tools that do not scale well with complexity when they are applied to realistic and complex problems. In this thesis we have been inspired by the work of Lee & Mumford (2003) and Dean (2006) in formulating an alternative model. The resulting novel Dynamic Hierarchical Nonparametric Belief Propagation (DHNBP) framework is based on the principals of the MPF framework and is able to facilitate representation of spatiotemporal sequences of features in a Dynamic Markov Network. The DHNBP framework is a novel extension of the Nonparametric Belief Propagation framework by Sudderth (2006) into hierarchies and time. In this report we provide algorithms for implementation, however, the DHNBP framework still has openended aspects that require further research.
A Hierarchical Bayesian Language Model based on PitmanYor Processes
"... We propose a new hierarchical Bayesian ngram model of natural languages. Our model makes use of a generalization of the commonly used Dirichlet distributions called PitmanYor processes which produce powerlaw distributions more closely resembling those in natural languages. We show that an approxi ..."
Abstract
 Add to MetaCart
(Show Context)
We propose a new hierarchical Bayesian ngram model of natural languages. Our model makes use of a generalization of the commonly used Dirichlet distributions called PitmanYor processes which produce powerlaw distributions more closely resembling those in natural languages. We show that an approximation to the hierarchical PitmanYor language model recovers the exact formulation of interpolated KneserNey, one of the best smoothing methods for ngram language models. Experiments verify that our model gives cross entropy results superior to interpolated KneserNey and comparable to modified KneserNey. 1