Keyword search in decentralized storage systems

Abstract

The emerging decentralized storage systems (DSSs), such as InterPlanetary File System (IPFS), Storj, and Sia, provide people with a new storage model. Instead of being centrally managed, the data are sliced up and distributed across the nodes of the network. Furthermore, each data object is uniquely identified by a cryptographic hash (ObjectId) and can only be retrieved by ObjectId. Compared with the search functions provided by the existing centralized storage systems, the application scenarios of the DSSs are subject to certain restrictions. In this paper, we first apply decentralized B+Tree and HashMap to the DSSs to provide keyword search. Both indexes are kept in blocks. Since these blocks may be scattered on multiple nodes, we ensure that all operations involve as few blocks as possible to reduce network cost and response time. In addition, the version control and version merging algorithms are designed to effectively organize the indexes and facilitate data integration. The experimental results prove that our indexes have excellent availability and scalability.