Impact on chunk size on deduplication and disk prefetch

Abstract

CAS (Content Addressable Storage) systems reduce total volume of virtual disk with deduplication technique. The effects of deduplication has been evaluated and confirmed in some papers. Most evaluations, however, were achieved by small chunk size (4KB-8KB) and did not care about I/O optimization (disk prefetch) on a real usage. Effective disk prefetch is larger than the chunk size and causes many CAS operations. Furthermore, previous evaluations did not care about ratio of effective data in a chunk. The ratio is improved by block reallocation of file system, which considers access profile. Chunk size should be decided by considering these effects on a real usage. This paper evaluates effectiveness of deduplication on a large chunk of CAS system which considers the optimization for disk prefetch and effective data in a chunk. The optimization was achieved for boot procedure, because it was a mandatory operation on any operating systems. The results showed large chunk (256KB) was effective on booting Linux and could maintain the effect of deduplication. © 2012 Springer-Verlag GmbH.