## Beyond streams and graphs: Dynamic tensor analysis (2006)

### Cached

### Download Links

- [www.cs.cmu.edu]
- [www.cs.cmu.edu]
- [www.cs.cmu.edu]
- [pdf.aminer.org]
- DBLP

### Other Repositories/Bibliography

Venue: | In KDD |

Citations: | 77 - 13 self |

### BibTeX

@INPROCEEDINGS{Sun06beyondstreams,

author = {Jimeng Sun and Dacheng Tao and Christos Faloutsos},

title = {Beyond streams and graphs: Dynamic tensor analysis},

booktitle = {In KDD},

year = {2006},

pages = {374--383}

}

### Years of Citing Articles

### OpenURL

### Abstract

How do we find patterns in author-keyword associations, evolving over time? Or in DataCubes, with product-branchcustomer sales information? Matrix decompositions, like principal component analysis (PCA) and variants, are invaluable tools for mining, dimensionality reduction, feature selection, rule identification in numerous settings like streaming data, text, graphs, social networks and many more. However, they have only two orders, like author and keyword, in the above example. We propose to envision such higher order data as tensors, and tap the vast literature on the topic. However, these methods do not necessarily scale up, let alone operate on semi-infinite streams. Thus, we introduce the dynamic tensor analysis (DTA) method, and its variants. DTA provides a compact summary for high-order and high-dimensional data, and it also reveals the hidden correlations. Algorithmically, we designed DTA very carefully so that it is (a) scalable, (b) space efficient (it does not need to store the past) and (c) fully automatic with no need for user defined parameters. Moreover, we propose STA, a streaming tensor analysis method, which provides a fast, streaming approximation to DTA. We implemented all our methods, and applied them in two real settings, namely, anomaly detection and multi-way latent semantic indexing. We used two real, large datasets, one on network flow data (100GB over 1 month) and one from DBLP (200MB over 25 years). Our experiments show that our methods are fast, accurate and that they find interesting patterns and outliers on the real datasets. 1.

### Citations

3648 | The Anatomy of a Large-Scale Hypertextual Web Search Engine
- Brin, Page
- 1998
(Show Context)
Citation Context ... where we can apply association rules [2] or “Ratio Rules” [21]; 3) the web, where both rows and columns are pages, and links correspond to edges between them; then we can apply HITS [19] or pageRank =-=[3]-=- to find hubs, authorities and influential nodes; all of them are identical or closely related to eigen analysis or derivatives; 4) social networks, and in fact, any graph (with un-labelled edges): pe... |

2999 | Authoritative sources in a hyperlinked environment
- Kleinberg
- 1999
(Show Context)
Citation Context ...roducts matrices, where we can apply association rules [2] or “Ratio Rules” [21]; 3) the web, where both rows and columns are pages, and links correspond to edges between them; then we can apply HITS =-=[19]-=- or pageRank [3] to find hubs, authorities and influential nodes; all of them are identical or closely related to eigen analysis or derivatives; 4) social networks, and in fact, any graph (with un-lab... |

2300 |
Principal Component Analysis
- Jolliffe
- 1986
(Show Context)
Citation Context ...ate the rank Ri. In general the rank can be different on each mode or can change over time. The idea is to keep ‹ the 2 smallest rank Ri such that the energy ratio �Si�F �X�F , is above the threshold =-=[15]-=-. In all experiments, we set the threshold as 0.9 unless mentioned otherwise. Complexity: The space consumption for incremental algorithm is QM i=1 Ni + PM i=1 Ni×Ri + PM i=1 Ri. The dominant factor i... |

1397 |
Adaptive Filter Theory
- Haykin
- 1991
(Show Context)
Citation Context ...ix Ud is computed by diagonalizing Cd. when historical tensors have the same weights as the new tensor X. Note that forgetting factor is a well-known trick in signal processing and adaptive filtering =-=[11]-=-. Update Rank: One thing missing from Figure 7 is how to update the rank Ri. In general the rank can be different on each mode or can change over time. The idea is to keep ‹ the 2 smallest rank Ri suc... |

511 | Parallel Multilevel k-way Partition Scheme for Irregular Graphs
- Karypis, Kumar
- 1999
(Show Context)
Citation Context ...dges): people are rows and columns; edges again correspond to nonzero entries in the adjacency matrix. The network value of a customer [13] has close ties to the first eigenvector; graph partitioning =-=[18]-=- is often done through matrix algebra (e.g. spectral clustering [16]); 5) streams and coevolving sequences can also be envisioned as matrices: each data source (sensor) corresponds to a row, and each ... |

363 |
Analysis of individual differences in multidimensional scaling via an n-way generalization of "Eckart-Young" decomposition
- Carroll, Chang
- 1970
(Show Context)
Citation Context ...r analysis have been applied successfully in many domains [5, 17, 29]. Powerful tools have been proposed, including Tucker decomposition [27], parallel factor analysis [10] or canonical decomposition =-=[4]-=-, and bilinear PCA. Tensors have been recently used in machine vision research, for example by Shashua and Levin [26] for linear image coding, by Vasilescu and Terzopoulos [28] for face recognition. Y... |

299 |
Foundations of the Parafac procedure: Models and conditions for an explanatory multimodal factor analysis
- Harshman
(Show Context)
Citation Context ...s: Tensor algebra and multilinear analysis have been applied successfully in many domains [5, 17, 29]. Powerful tools have been proposed, including Tucker decomposition [27], parallel factor analysis =-=[10]-=- or canonical decomposition [4], and bilinear PCA. Tensors have been recently used in machine vision research, for example by Shashua and Levin [26] for linear image coding, by Vasilescu and Terzopoul... |

270 | Information-theoretic co-clustering
- Dhillon, Mallela, et al.
- 2003
(Show Context)
Citation Context ... as a graph. Analysis and mining of static graphs has attracted a lot of interest, with many graph partitioning methods, including METIS [18], spectral partitioning [16], informationtheoretic methods =-=[6]-=- and several variations. All these works focus on static graphs (2nd order tensors), while our emphasis is on time-evolving sequences of graphs and in general,ssequences of tensors of potentially even... |

267 | Mining Timechanging Data Streams
- Hulten, Spencer, et al.
- 2001
(Show Context)
Citation Context ...s; 4) social networks, and in fact, any graph (with un-labelled edges): people are rows and columns; edges again correspond to nonzero entries in the adjacency matrix. The network value of a customer =-=[13]-=- has close ties to the first eigenvector; graph partitioning [18] is often done through matrix algebra (e.g. spectral clustering [16]); 5) streams and coevolving sequences can also be envisioned as ma... |

266 | On clusterings — good, bad and spectral
- Kannan, Vempala, et al.
- 2000
(Show Context)
Citation Context ...o entries in the adjacency matrix. The network value of a customer [13] has close ties to the first eigenvector; graph partitioning [18] is often done through matrix algebra (e.g. spectral clustering =-=[16]-=-); 5) streams and coevolving sequences can also be envisioned as matrices: each data source (sensor) corresponds to a row, and each time-tick to a column. Then we can do multivariate analysis or SVD [... |

247 |
Personalized information delivery: An analysis of information filtering methods
- Foltz, Dumais
- 1992
(Show Context)
Citation Context ...presented as a matrix. Important applications of this view point include numerous settings like: 1) information retrieval, where the data can be turned into a document-term matrix, and then apply LSI =-=[9, 24]-=-; 2) market basket analysis, with customer-products matrices, where we can apply association rules [2] or “Ratio Rules” [21]; 3) the web, where both rows and columns are pages, and links correspond to... |

239 |
Some mathematical notes on three-mode factor analysis
- Tucker
- 1966
(Show Context)
Citation Context ... Tensor and Multilinear Analysis: Tensor algebra and multilinear analysis have been applied successfully in many domains [5, 17, 29]. Powerful tools have been proposed, including Tucker decomposition =-=[27]-=-, parallel factor analysis [10] or canonical decomposition [4], and bilinear PCA. Tensors have been recently used in machine vision research, for example by Shashua and Levin [26] for linear image cod... |

151 | Multilinear analysis of image ensembles: Tensorfaces
- Vasilescu, Terzopoulos
- 2002
(Show Context)
Citation Context ... canonical decomposition [4], and bilinear PCA. Tensors have been recently used in machine vision research, for example by Shashua and Levin [26] for linear image coding, by Vasilescu and Terzopoulos =-=[28]-=- for face recognition. Ye [31] presented the generalized low rank approximation which extends PCA from the vectors (1st-order tensors) into matrices (2nd order tensors). Ding and Ye [7] proposed an ap... |

121 |
Tomasz Imielinski, and Arun Swami. Mining Association Rules Between Sets of Items in Large Databases
- Agrawal
- 1993
(Show Context)
Citation Context ...ation retrieval, where the data can be turned into a document-term matrix, and then apply LSI [9, 24]; 2) market basket analysis, with customer-products matrices, where we can apply association rules =-=[2]-=- or “Ratio Rules” [21]; 3) the web, where both rows and columns are pages, and links correspond to edges between them; then we can apply HITS [19] or pageRank [3] to find hubs, authorities and influen... |

86 | Principal component analysis of three-mode data by means of alternating least squares algorithms. Psychometrika 45: 69–97
- PM, Leeuw
- 1980
(Show Context)
Citation Context ...ents (i.e., new tensors keep coming). Even if we want to apply OTA every time that a new tensor arrives, it is prohibitively expensive 3 Similar ideas have been proposed for 2nd- or 3rd order tensors =-=[22, 30, 12]-=-. U2 T Input: The tensors Xi| n i=1 ∈ R N1×...×NM The dimensionality of the output tensors Yi ∈ R R1×...×RM. Output: The projection matrix Ul| M l=1 ∈ R Nl×Rl constrained by �Ul� = I and the output te... |

67 |
Signal processing based on multilinear algebra
- Lathauwer
- 1997
(Show Context)
Citation Context ...focus has changed from object-oriented (1995) to stream (2004). 8. RELATED WORK Tensor and Multilinear Analysis: Tensor algebra and multilinear analysis have been applied successfully in many domains =-=[5, 17, 29]-=-. Powerful tools have been proposed, including Tucker decomposition [27], parallel factor analysis [10] or canonical decomposition [4], and bilinear PCA. Tensors have been recently used in machine vis... |

65 |
Identifying Representative Trends in Massive Time Series Data Sets Using Sketches
- Indyk, Koudas, et al.
- 2000
(Show Context)
Citation Context ... can also be envisioned as matrices: each data source (sensor) corresponds to a row, and each time-tick to a column. Then we can do multivariate analysis or SVD [25],“sketches” and random projections =-=[14]-=- to find patterns and outliers. The need for tensors: Powerful as they may be, matrixbased tools can handle neither of the two problems we stated in the beginning. The crux is that matrices have only ... |

64 | Generalized low rank approximation of matrices
- Ye
(Show Context)
Citation Context ...and bilinear PCA. Tensors have been recently used in machine vision research, for example by Shashua and Levin [26] for linear image coding, by Vasilescu and Terzopoulos [28] for face recognition. Ye =-=[31]-=- presented the generalized low rank approximation which extends PCA from the vectors (1st-order tensors) into matrices (2nd order tensors). Ding and Ye [7] proposed an approximation of [31]. Similar a... |

54 |
Linear image coding for regression and classification using the tensor-rank principle
- Shashua, Levin
- 2001
(Show Context)
Citation Context ...Tucker decomposition [27], parallel factor analysis [10] or canonical decomposition [4], and bilinear PCA. Tensors have been recently used in machine vision research, for example by Shashua and Levin =-=[26]-=- for linear image coding, by Vasilescu and Terzopoulos [28] for face recognition. Ye [31] presented the generalized low rank approximation which extends PCA from the vectors (1st-order tensors) into m... |

49 | Higher-order web link analysis using multilinear algebra - Kolda, Bader, et al. - 2005 |

26 | TriCluster: An Effective Algorithm for Mining Coherent Clusters
- Zhao, Zaki
(Show Context)
Citation Context ... with edge-type being the 3rd mode. Over time, we have multiple 3rd order tensors, which are still within the reach of our upcoming tools. 3) Microarray data, with gene expressions evolving over time =-=[32]-=-. Here we have genes-proteins over time, and we record the expression level: a series of 2nd order tensors. 4) All OLAP and DataCube applications: customer-product-branch sales data is a 3rd order ten... |

25 | A randomized algorithm for a tensor-based generalization of the singular value decomposition. Linear algebra and its applications
- Drineas, Mahoney
(Show Context)
Citation Context ...posed an approximation of [31]. Similar approach is also proposed in [12]. Xu et al. [30] formally presented the tensor representation for PCA and applied it for face recognition. Drineas and Mahoney =-=[8]-=- showed how to approximate the tensor SVD using biased sampling. These methods do one or more of the following assumptions: the dataset is dense, or static. We are interested in sparse, streams of ten... |

25 | Concurrent subspace analysis
- Xu, Yan, et al.
(Show Context)
Citation Context ...ents (i.e., new tensors keep coming). Even if we want to apply OTA every time that a new tensor arrives, it is prohibitively expensive 3 Similar ideas have been proposed for 2nd- or 3rd order tensors =-=[22, 30, 12]-=-. U2 T Input: The tensors Xi| n i=1 ∈ R N1×...×NM The dimensionality of the output tensors Yi ∈ R R1×...×RM. Output: The projection matrix Ul| M l=1 ∈ R Nl×Rl constrained by �Ul� = I and the output te... |

22 |
Hisao Tamaki, and Santosh Vempala. Latent semantic indexing: A probabilistic analysis
- Papadimitriou, Raghavan
- 1998
(Show Context)
Citation Context ...presented as a matrix. Important applications of this view point include numerous settings like: 1) information retrieval, where the data can be turned into a document-term matrix, and then apply LSI =-=[9, 24]-=-; 2) market basket analysis, with customer-products matrices, where we can apply association rules [2] or “Ratio Rules” [21]; 3) the web, where both rows and columns are pages, and links correspond to... |

15 | Two-dimensional singular value decomposition (2dsvd) for 2d maps and images
- Ding, Ye
- 2005
(Show Context)
Citation Context ...ind the projection matrix for that mode. This process potentially needs performing more than one iteration. In practice, this algorithm converges in a small number of iterations. In fact, Ding and Ye =-=[7]-=- shows the nearoptimality of a similar algorithm for 2nd order tensors. Therefore, for time-critical applications, single iteration is usually sufficient to obtain the near-optimal projection matrices... |

15 |
An approach to n-mode components analysis
- Kapteyn, Neudecker, et al.
- 1986
(Show Context)
Citation Context ...focus has changed from object-oriented (1995) to stream (2004). 8. RELATED WORK Tensor and Multilinear Analysis: Tensor algebra and multilinear analysis have been applied successfully in many domains =-=[5, 17, 29]-=-. Powerful tools have been proposed, including Tucker decomposition [27], parallel factor analysis [10] or canonical decomposition [4], and bilinear PCA. Tensors have been recently used in machine vis... |

7 |
Partha Niyogi. Tensor subspace analysis
- He, Cai
- 2005
(Show Context)
Citation Context ...ents (i.e., new tensors keep coming). Even if we want to apply OTA every time that a new tensor arrives, it is prohibitively expensive 3 Similar ideas have been proposed for 2nd- or 3rd order tensors =-=[22, 30, 12]-=-. U2 T Input: The tensors Xi| n i=1 ∈ R N1×...×NM The dimensionality of the output tensors Yi ∈ R R1×...×RM. Output: The projection matrix Ul| M l=1 ∈ R Nl×Rl constrained by �Ul� = I and the output te... |

6 |
Data streams: algorithms and applications, volume 1. Foundations and Trends
- Muthukrishnan
- 2005
(Show Context)
Citation Context ... static. We are interested in sparse, streams of tensors, like the IP traffic matrices over time. Stream and Graph Mining: Data streams have been extensively studied in recent years. A recent surveys =-=[23]-=- have discussed many data streams algorithms. Among them, Papadimitriou et al. [25] proposes an online algorithm that summarizes the multiple streams incrementally. Our STA algorithm has a similar fla... |

3 |
Jimeng Sun, and Christos Faloutsos, Streaming pattern discovery in multiple time-series
- Papadimitriou
(Show Context)
Citation Context ...]); 5) streams and coevolving sequences can also be envisioned as matrices: each data source (sensor) corresponds to a row, and each time-tick to a column. Then we can do multivariate analysis or SVD =-=[25]-=-,“sketches” and random projections [14] to find patterns and outliers. The need for tensors: Powerful as they may be, matrixbased tools can handle neither of the two problems we stated in the beginnin... |

1 |
Yannis Kotidis, and Christos Faloutsos. Quantifiable data mining using ratio rules
- Korn, Labrinidis
- 2000
(Show Context)
Citation Context ...e the data can be turned into a document-term matrix, and then apply LSI [9, 24]; 2) market basket analysis, with customer-products matrices, where we can apply association rules [2] or “Ratio Rules” =-=[21]-=-; 3) the web, where both rows and columns are pages, and links correspond to edges between them; then we can apply HITS [19] or pageRank [3] to find hubs, authorities and influential nodes; all of the... |

1 |
Monitoring Thermal Processes by NMR Technology
- Viereck, Dyrby, et al.
- 2006
(Show Context)
Citation Context ...focus has changed from object-oriented (1995) to stream (2004). 8. RELATED WORK Tensor and Multilinear Analysis: Tensor algebra and multilinear analysis have been applied successfully in many domains =-=[5, 17, 29]-=-. Powerful tools have been proposed, including Tucker decomposition [27], parallel factor analysis [10] or canonical decomposition [4], and bilinear PCA. Tensors have been recently used in machine vis... |