Efficient Hashing of Sparse Virtual Disks

Abstract

Verifying the integrity of a file is a fundamental operation in file transfer. Common tools compute a short hash value that is sent along with the file, but computing this value requires going over the entire file and if the file is huge, then this process is slow. We introduce blkhash - a novel hash algorithm optimized for disk images, that is up to 4 orders of magnitude faster than commonly used tools. We implemented a new command line tool and library that can be used in the virtualization space for verifying storage management operations. Our approach can significantly contribute to use cases such as: (1) Very fast computing of virtual disk hash value in software defined storage, (2) Verifying an entire disk image content as part of a supply chain integrity verification or in the context of confidential computing.