A memory-efficient data redistribution algorithm

Abstract

Many memory-bound distributed applications require frequent redistribution of data. Pinar and Hendrickson investigated two families of memory-limited redistribution algorithms. The first family has many advantages, but fails on certain inputs, and, if not implemented carefully, may lead to an explosion in the number of local data copies. The second family eliminates the possibility of failure at the expense of considerable additional overhead. We carefully analyze these algorithms and develop a modified method that potentially combines advantages of each. The resulting algorithm has been implemented in MADRE and experiments reveal its performance to be superior to that of other MADRE algorithms in most cases. © 2009 Springer Berlin Heidelberg.