Exploring I/O virtualization data paths for MPI applications in a cluster of VMs: A networking perspective

Abstract

Nowadays, seeking optimized data paths that can increase I/O throughput in Virtualized environments is an intriguing task, especially in a high-performance computing context. This study endeavors to address this issue by evaluating methods for optimized network device access using scientific applications and micro-benchmarks. We examine the network performance bottlenecks that appear in a Cluster of Xen VMs using both generic and intelligent network adapters. We study the network behavior of MPI applications. Our goal is to: (a) explore the implications of alternative data paths between applications and network hardware and (b) specify optimized solutions for scientific applications that put pressure on network devices. To monitor the network load and the applications' total throughput we build a custom testbed using different network configurations. We use the Xen bridge mechanism and I/O Virtualization techniques and examine the trade-offs. Preliminary results show that a combination of these techniques is essential to overcome network virtualization overheads and achieve near-native performance. © 2011 Springer-Verlag Berlin Heidelberg.