Application of Probabilistic Common Set on an Open World Set for Vertical Federated Learning

Abstract

Vertical federated learning (VFL) is a distributed machine learning technology that is suitable for model building in organizations across different industries. It enables the identification of a common set of data that co-occur across organizations. However, VFL uses private set intersection (PSI) protocols, which requires making all data shareable, and satisfying the data minimization principle in the General Data Protection Regulation is difficult. To mitigate noncompliance in privacy regulations, we propose a new VFL method that uses horizontal federated learning to identify the common set instead of PSI. The method consists of two concepts: The first is to use a common data structure between organizations to avoid using PSI. The second is to identify the common set from machine learning classifiers of unseen data of a certain class. Our proposed method considers that the data labeled as the desired class is unseen data and it is not in the common set. Experimental results show that the F-measure is 0.8 or higher in 40% of the common set ratios.