An Evaluation of Storage Alternatives for Service Interfaces Supporting a Decentralized AI Marketplace

Abstract

Given the exploding interest in generative AI and the concern that a few companies like Microsoft will monopolize access to such models, we address this centralization risk in the context of a DApp that matches buyers and sellers of various AI services. A key question for a decentralized marketplace is where and how to store the metadata that specifies the services' properties in human and machine-readable formats. Having one or a few actors controlling access to that data constitutes undesirable centralization. We explore data storage alternatives to ensure decentralization, equitable match-making, and efficiency. Classifying decentralized storage alternatives as simple peer-to-peer replication, replication governed by a permissionless consensus, and replication governed by a private consensus, we select an exemplar for each category: IPFS, Tendermint Cosmos and Hyperledger Fabric. We conduct experiments on performance and find that read and write speeds are fastest for IPFS, about two times slower for Tendermint and slowest for Hyperledger. Writing using IPFS and Tendermint takes significantly longer than reading, and finally, specifically with IPFS, write speeds strongly depend on configuration. Given these results and the properties of the storage technologies, we conclude that simple peer-to-peer storage is the best option for the proposed AI marketplace.