On the Dirichlet Prior and Bayesian Regularization

On the Dirichlet Prior and Bayesian Regularization (2002)

Cached

Download Links

Other Repositories/Bibliography

DBLP

by Harold Steck , Tommi S. Jaakkola

Venue:	In Advances in Neural Information Processing Systems 15
Citations:	20 - 2 self

Versions

Version History

Metadata Version 2

User correction supplied by SystemCorrections

Datum	Value	Source
TITLE	On the Dirichlet Prior and Bayesian Regularization	user correction - Legacy Corrections
AUTHOR NAME	Harold Steck	user correction - Legacy Corrections
AUTHOR NAME	Tommi S. Jaakkola	user correction - Legacy Corrections
ABSTRACT	A common objective in learning a model from data is to recover its network structure, while the model parameters are of minor interest. For example, we may wish to recover regulatory networks from high-throughput data sources. In this paper we examine how Bayesian regularization using a Dirichlet prior over the model parameters affects the learned model structure in a domain with discrete variables. Surprisingly, a weak prior in the sense of smaller equivalent sample size leads to a strong regularization of the model structure (sparse graph) given a sufficiently large data set. In particular, the empty graph is obtained in the limit of a vanishing strength of prior belief. This is diametrically opposite to what one may expect in this limit, namely the complete graph from an (unregularized) maximum likelihood estimate. Since the prior affects the parameters as expected, the prior strength balances a "trade-off" between regularizing the parameters or the structure of the model. We demonstrate the benefits of optimizing this trade-off in the sense of predictive accuracy.	user correction - Legacy Corrections
YEAR	2002	INFERENCE
VENUE	In Advances in Neural Information Processing Systems 15	INFERENCE
VENUE TYPE	CONFERENCE	INFERENCE
PAGES	697--704	INFERENCE
VOLUME	15	INFERENCE
CITATIONS	9 found	ParsCit 1.0