Taking on the Curse of Dimensionality in Joint Distributions Using Neural Networks (2000)
| Venue: | IEEE Transactions on Neural Networks, special issue on Data Mining and Knowledge Discovery |
| Citations: | 10 - 4 self |
BibTeX
@ARTICLE{Bengio00takingon,
author = {Samy Bengio and Yoshua Bengio},
title = {Taking on the Curse of Dimensionality in Joint Distributions Using Neural Networks},
journal = {IEEE Transactions on Neural Networks, special issue on Data Mining and Knowledge Discovery},
year = {2000},
volume = {11},
pages = {200--0}
}
Bookmark
 |
 |
 |
 |
 |
 |
OpenURL
Abstract
|The curse of dimensionality is severe when modeling high-dimensional discrete data: the number of possible combinations of the variables explodes exponentially. In this paper we propose a new architecture for modeling high-dimensional data that requires resources (parameters and computations) that grow at most as the square of the number of variables, using a multi-layer neural network to represent the joint distribution of the variables as the product of conditional distributions. The neural network can be interpreted as a graphical model without hidden random variables, but in which the conditional distributions are tied through the hidden units. The connectivity of the neural network can be pruned by using dependency tests between the variables (thus reducing signicantly the number of parameters). Experiments on modeling the distribution of several discrete data sets show statistically signicant improvements over other methods such as naive Bayes and comparable Bayesian networks,...
Citations
No citations identified.
