Bundlefly: A low-diameter topology for multicore fiber

Abstract

High-performance computing (HPC) systems keep increasing in size and bandwidth, thus requiring larger and higher-bandwidth interconnection networks. The race to exascale just exacerbated this trend. The resulting longer average distance and more links between modules makes the use of optical fiber mandatory. However, the system meets the challenge of cable packaging complexity, cable tolerance, and cable maintainability. Splitter cable, like multi-core fiber (MCF), is a new and cost-effective approach that has the potential to replace a bundle of fibers between any pairs of modules with a single cable, thus lowering the packaging complexity and enhancing the maintainability. To the best of our knowledge, we are the first to formally study the problem of building a cost-effective HPC network topology using multicore fiber. In this paper, a new diameter-3 topology is proposed, namely Bundlefly. It achieves a flexible tradeoff between intra-module radixes and inter-module radixes of routers with merely moderate radix to build a diameter-3 exascale interconnection network. It is suitable for the use of multi-core fiber for the requirement of inter-module bandwidth and cable packaging complexity. We analyze the properties of Bundlefly and present effective routing algorithms. We simulate and analyze the performance of Bundlefly against state-of-the-art topologies. The results show that Bundlefly with flexible configurations can achieve better performance than most existing topologies.