## Applying Discrete PCA in Data Analysis (2004)

### Cached

### Download Links

Citations: | 54 - 10 self |

### BibTeX

@MISC{Buntine04applyingdiscrete,

author = {Wray Buntine and Aleks Jakulin},

title = { Applying Discrete PCA in Data Analysis},

year = {2004}

}

### OpenURL

### Abstract

Methods for analysis of principal components in discrete data have existed for some time under various names such as grade of membership modelling, probabilistic latent semantic analysis, and genotype inference with admixture. In this paper we explore a number of extensions to the common theory, and present some application of these methods to some common statistical tasks. We show that these methods can be interpreted as a discrete version of ICA. We develop a hierarchical version yielding components at different levels of detail, and additional techniques for Gibbs sampling. We compare the algorithms on a text prediction task using support vector machines, and to information retrieval.

### Citations

2365 | Latent dirichlet allocation
- Blei, Ng, et al.
- 2003
(Show Context)
Citation Context ...cs, genotype inference using admixtures ∗ In Uncertainty in AI, Banff, Canada, July 2004. (Pritchard et al., 2000), probabilistic latent semantic indexing (Hofmann, 1999) latent Dirichlet allocation (=-=Blei et al., 2003-=-), and multiple aspect modelling for document analysis (Minka & Lafferty, 2002). These methods are equivalent, ignoring statistical methodology and notation. Note the representation of (Pritchard et a... |

1493 | Topographic independent component analysis
- Hyvarinen, Hoyer, et al.
(Show Context)
Citation Context ...text prediction task using support vector machines, and to information retrieval. 1 INTRODUCTION Principal component analysis (PCA) latent semantic indexing, and independent component analysis (ICA) (=-=Hyvärinen et al., 2001-=-) are key methods in the statistical engineering toolbox. They have a long history, are used in many different ways, and under different names. They were primarily developed in the engineering communi... |

1440 |
Making large-scale svm learning practical
- Joachims
- 1999
(Show Context)
Citation Context ...ommon use for PCA and ICA, and as a classification tool. For this, we used the 20 newsgroups collection described previously as well as the Reuters-21578 collection3 . We employed the SVM light V5.0 (=-=Joachims, 1999-=-) classifier with default settings. For classification, we added the class as a distinct multinomial (cf. Section 3.1) for the training data and left it empty for the test data, and then predicted the... |

784 | Probabilistic latent semantic indexing
- Hofmann
- 1999
(Show Context)
Citation Context ...sciences, demographics and medical informatics, genotype inference using admixtures ∗ In Uncertainty in AI, Banff, Canada, July 2004. (Pritchard et al., 2000), probabilistic latent semantic indexing (=-=Hofmann, 1999-=-) latent Dirichlet allocation (Blei et al., 2003), and multiple aspect modelling for document analysis (Minka & Lafferty, 2002). These methods are equivalent, ignoring statistical methodology and nota... |

410 |
Inference of population structure using multilocus genotype data
- Pritchard, Stephens, et al.
- 2000
(Show Context)
Citation Context ...ership (Woodbury & Manton, 1982) used for instance in the social sciences, demographics and medical informatics, genotype inference using admixtures ∗ In Uncertainty in AI, Banff, Canada, July 2004. (=-=Pritchard et al., 2000-=-), probabilistic latent semantic indexing (Hofmann, 1999) latent Dirichlet allocation (Blei et al., 2003), and multiple aspect modelling for document analysis (Minka & Lafferty, 2002). These methods a... |

202 | Language modeling for information retrieval
- Croft, Lafferty
- 2003
(Show Context)
Citation Context ... 2: football fans in Germany. In Tables 3 and 4, five discrete PCA results are first, then five TF-IDF results. This inference method is in the spirit of language modelling for information retrieval (=-=Croft & Lafferty, 2003-=-) and it is clear that topical content is being retrieved. Note that articles retrieved using the mean field approximation to perform inference were poor, as expected from Section 3.5. Generally some ... |

197 | Pairwise data clustering by deterministic annealing
- HOFMANN, BUHMANN
- 1997
(Show Context)
Citation Context ...problem follow some of the usual approaches in the community, albeit with considerable sophistication: • Annealed maximum likelihood (Hofmann, 1999), best viewed in terms of its clustering precursor (=-=Hofmann & Buhmann, 1997-=-), • Gibbs sampling on w, m and Ω in turn using a full probability distribution (Pritchard et al., 2000), • mean field methods (Blei et al., 2003), and • expectation propagation (EP, like so-called ca... |

110 | Propagation algorithms for Variational Bayesian learning
- Ghahramani, Beal
- 2001
(Show Context)
Citation Context ...plement. The mean field version now has dual parameters giving a Beta and Dirichlet approximation to the posteriors of qk and nk respectively. Using the mean field formulation of Ghahramani and Beal (=-=Ghahramani & Beal, 2000-=-) presented for discrete PCA in (Buntine, 2002), the development is tedious but straightforward, and has the same form as its non-hierarchical version. 3.5 INFERENCE ON NEW DATA A typical use of the m... |

110 | Expectation-propagation for the generative aspect model
- Minka, Lafferty
- 2002
(Show Context)
Citation Context ...nada, July 2004. (Pritchard et al., 2000), probabilistic latent semantic indexing (Hofmann, 1999) latent Dirichlet allocation (Blei et al., 2003), and multiple aspect modelling for document analysis (=-=Minka & Lafferty, 2002-=-). These methods are equivalent, ignoring statistical methodology and notation. Note the representation of (Pritchard et al., 2000) is completely different, thus one also needs to translate notations.... |

30 | Analyzing attribute dependencies
- Jakulin, Bratko
- 2003
(Show Context)
Citation Context ...en achieved. An inves2 See http://www.ai.mit.edu/∼jrennie/20Newsgroups/. 0.0 0.2 0.4 0.6 0.8 D B−B T−B T−T Figure 1: Box Plots for the Two Models tigation into the interactions here using methods of (=-=Jakulin & Bratko, 2003-=-) revealed that in the hierarchical model much of the interaction is up and down the hierarchy. Children and parents can have some dependence. 6 CLASSIFICATION EXPERIMENTS We tested the use of discret... |

2 |
Variational extensions to EM and multinomial PCA. ECML
- Buntine
- 2002
(Show Context)
Citation Context ... Linux that was used in these experiments and is available under the GNU GPL license from him. 2 THE BASIC MODEL A good introduction to these models from a number of viewpoints is (Blei et al., 2003; =-=Buntine, 2002-=-). They are directly analogous to the Gaussian model of principal component analysis (Buntine, 2002). The simplest version consists of a linear admixture of different multinomials, and can be thought ... |