Dynamic block partitioning strategy for cloud-backed file systems

Abstract

Cloud storage services, like S3, has drawn increasing popularity as primary data storage repository for users and enterprises because data can be conveniently accessed and shared through simplified web service based interface. However, users still desire the full use of the performance and capacity of their purchased storage resources, and to retain the POSIX I/O interface for accommodating general applications on local systems. Therefore, there are increasing research interests in cloud-backed file systems that aim to achieve the best of both worlds by connecting POSIX-interface local file system with a remote cloud storage system. However, due to the differences between the two types of storage systems, many challenges arise. One of them is to support efficient random file access on the object cloud storage system. Several frameworks have proposed to partition a file into fixed-size blocks and store them as individual objects on cloud for partial data access. But the setting of block size is not trivial and can have significant performance impact. Larger block size may contain data that is not requested by users. Smaller block may require more data access requests and lower network bandwidth utilization. To overcome the demerit of a static fixed-size strategies, this paper proposes a dynamically block partitioning strategy that will merge and split blocks on-the-fly according to the I/O access pattern of a file. Our simulation results show that our strategy can improve I/O throughput by 37% to 321% comparing to the static strategy using different fixed-size settings.