Coflow: A networking abstraction for cluster applications

Abstract

Cluster computing applications - frameworks like MapReduce and user-facing applications like search platforms - have application-level requirements and higher-level abstractions to express them. However, there exists no networking abstraction that can take advantage of the rich semantics readily available from these data parallel applications. We propose coflow, a networking abstraction to express the communication requirements of prevalent data parallel programming paradigms. Coflows make it easier for the applications to convey their communication semantics to the network, which in turn enables the network to better optimize common communication patterns. Copyright © 2012 by the Association for Computing Machinery, Inc.