HITP: A transmission protocol for scalable high-performance distributed storage

Abstract

In many application and research areas there is a trend towards petascale processing and archiving of scientific measurement results and derived data structures. As a consequence, there is an urgent need for disruptive technologies that revolutionize the manner in which we integrate, coordinate, and distribute computing and storage of vast amounts of data. This paper discusses the INCA architecture, a system that aims at providing a sustainable solution to the management problem of data storage in very large-scale distributed systems. INCA provides a solution where 'intelligence' in a globally scalable system is put in the middleware, where the lower level deals with local problems only, while all other properties are taken care of at the upper layers. In particular, we focus on a new, efficient transport protocol (referred to as HITP) that is specifically designed for bulk data transfer over high-speed (optical) network connections. We present performance results for basic transmission functionality, and show that HITP is capable of achieving higher throughput and lower latency than existing transport protocols. © 2008 Springer-Verlag Berlin Heidelberg.