Accelerating all-sources bfs metrics on multi-core clusters for large-scale complex network analysis

Abstract

All-Sources BFS (AS-BFS) is the main building block in a variety of complex network metric algorithms, such as the average path length and the betweenness centrality. However, AS-BFS calculations involve as many full BFS traversals as the total number of vertices, rendering AS-BFS impractical on commodity systems for real-world graphs with millions of vertices and links. In this paper we present our experience with the acceleration of AS-BFS graph metrics on multi-core HPC clusters by outlining hybrid coarse-grain parallel algorithms for computing the average path-length, the diameter and the betweenness centrality of complex networks in a lock-free fashion. We report speedups of up to 171× on a heterogeneous cluster of 12-core Intel Xeon and 32-core AMD Opteron multi-core nodes; as well as resource utilizations of up to 75%.