Abstract

Currently, most research on nonnegative matrix factorization (NMF) focus on 2-factor X = FG T factorization. We provide a systematic analysis of 3-factor X = FSG T NMF. While unconstrained 3-factor NMF is equivalent to unconstrained 2-factor NMF, constrained 3factor NMF brings new features to constrained 2-factor NMF. We study the orthogonality constraint because it leads to rigorous clustering interpretation. We provide new rules for updating F,S,G and prove the convergence of these algorithms. Experiments on 5 datasets and a real world case study are performed to show the capability of bi-orthogonal 3-factor NMF on simultaneously clustering rows and columns of the input data matrix. We provide a new approach of evaluating the quality of clustering on words using class aggregate distribution and multi-peak distribution. We also provide an overview of various NMF extensions and examine their relationships.

Citations

698	Learning the parts of objects by non-negative matrix factorization - Lee, Seung - 1999
450	Using Linear Algebra for Intelligent Information Retrieval - Berry, Dumais, et al. - 1995
375	Probabilistic latent semantic analysis - Hofmann - 1999
287	Objective criteria for the evaluation of clustering methods - Rand - 1971
231	Concept decomposition for large sparse text data using clustering - Dhillon, Modha
220	Co-clustering documents and words using bipartite spectral graph partitioning - Dhillon - 2001
200	Non-negative matrix factorization with sparseness constraints - Hoyer - 2004
152	Positive matrix factorization: a nonnegative factor model with optimal utilization of error estimates of data values - Paatero, Tapper - 1994
133	A toolkit for statistical language modeling, text retrieval, classi and clustering. http://www.cs.cmu.edu/mccallum/bow - Bow - 1996
117	Y.: Document clustering based on non-negative matrix factorization - Xu, Liu, et al. - 2003
116	Spectral relaxation for k-means clustering - Zha, Ding, et al. - 2001
93	Learning spatially localized, parts-based representation - Li, Hou, et al.
65	Bipartite graph partitioning and data clustering - Zha, He, et al. - 2001
60	H.D.: On the equivalence of nonnegative matrix factorization and spectral clustering - Ding, He, et al. - 2005
58	Concept indexing: a fast dimensionality reduction algorithm with applications IO document retrieval & categorization, ClKM - Karypis, Sam - 2000
57	WebACE: A web agent for document categorization and exploration - Han, Boley, et al. - 1998
56	Mesirov, “Metagenes and molecular pattern discovery using matrix factorization - Brunet, Tamayo, et al. - 2004
46	A study of the comparability of external criteria for hierarchical cluster analysis - Milligan, Cooper - 1986
44	Empirical and Theoretical Comparisons of Selected Criterion Functions for Document Clustering - Zhao, Karypis - 2004
41	Multiplicative updates for nonnegative quadratic programming in support vector machines - Sha, Saul, et al. - 2007
38	Generalizing Discriminant Analysis Using the Generalized Singular Value Decomposition - Howland, Park - 2004
28	Towards informatic analysis of syslogs - Stearley - 2004
27	Principal direction divisive partitioning. Data mining and knowledge discovery - Boley - 1998
25	A general model for clustering binary data - Li - 2005
22	Document clustering via adaptive subspace iteration - Li, Ma, et al. - 2004
20	PS: Co-clustering by block value decomposition - Long, Zhang, et al.
18	Summarizing video using non-negative similarity matrix factorization - Cooper, Foote - 2002
14	K-means clustering and principal component analysis - Ding, He - 2004
13	Nonnegative matrix factorization and probabilistic latent semantic indexing: Equivalence chi-square statistic and a hybrid method - Ding, Li, et al.
12	An Integrated Framework on Mining Logs Files for Computing System Management - Li, Liang, et al.
11	CLSI: A flexible approximation scheme from clustered term-document matrices - Gallopoulos, Zeimpekis - 2005
9	Unsupervised learning: self-aggregation in scaled principal component space - Ding, He, et al.
8	Algorithms for non-negatvie matrix factorization - Lee, Seung
7	Convex and Semi-Nonnegative Matrix Factorizations for Clustering and Low-Dimension Representation - Ding, Li, et al. - 2006
6	Specification: Common base event - Chessell - 2003