A field test of a federated learning/federated analytic blockchain network implementation in an HPC environment

Abstract

The rapid upswing in interest in federated learning (FL) and federated analytics (FA) architectures has corresponded with the rapid increase in commercial AI software products, ranging from face detection and language translation to connected IOT devices, smartphones, and autonomous vehicles equipped with high-resolution sensors. However, the traditional client-server model does not readily address questions of data ownership, privacy, and data location in the context of the multiple datasets required for machine learning. In this paper, we report on a pilot distributed ledger and smart contract network model, designed to track analytic jobs in an HPC supercomputing environment. The test system design integrates the FL/FA model into a blockchain-based network architecture, wherein the test system records interactions with the global server and blockchain network. The design goal is to create a secure audit trail of supercomputer analytic operations and the ability to securely federate those operations across multiple supercomputer deployments. As there are still relatively few real-world applications of FL/FA models and blockchain networks in use, our system design, test deployment, and sample code are intended to provide interested researchers with exploratory tools for future research.