Results 1 
2 of
2
Spectral Analysis for BillionScale Graphs: Discoveries and Implementation
"... Abstract. Given a graph with billions of nodes and edges, how can we find patterns and anomalies? Are there nodes that participate in too many or too few triangles? Are there closeknit nearcliques? These questions are expensive to answer unless we have the first several eigenvalues and eigenvector ..."
Abstract

Cited by 31 (13 self)
 Add to MetaCart
(Show Context)
Abstract. Given a graph with billions of nodes and edges, how can we find patterns and anomalies? Are there nodes that participate in too many or too few triangles? Are there closeknit nearcliques? These questions are expensive to answer unless we have the first several eigenvalues and eigenvectors of the graph adjacency matrix. However, eigensolvers suffer from subtle problems (e.g., convergence) for large sparse matrices, let alone for billionscale ones. We address this problem with the proposed HEIGEN algorithm, which we carefully design to be accurate, efficient, and able to run on the highly scalable MAPREDUCE (HADOOP) environment. This enables HEIGEN to handle matrices more than 1000 × larger than those which can be analyzed by existing algorithms. We implement HEIGEN and run it on the M45 cluster, one of the top 50 supercomputers in the world. We report important discoveries about nearcliques and triangles on several realworld graphs, including a snapshot of the Twitter social network (38Gb, 2 billion edges) and the “YahooWeb ” dataset, one of the largest publicly available graphs (120Gb, 1.4 billion nodes, 6.6 billion edges). 1