Dynamic Graph Segmentation for Deep Graph Neural Networks

Abstract

We present Deep network Dynamic Graph Partitioning (DDGP), a novel algorithm for optimizing the division of large graphs for mixture of expert graph neural networks. Our work is motivated from the observation that real world graphs suffer from spatial concept drift, which is detrimental to neural network training. We answer the question of how we can divide a graph, with vertices in each subgraph sharing a similar distribution, so that an expert network trained over each subgraph may yield the best learning outcome. DDGP is a two pronged algorithm that consists of cluster merging, followed by cluster boundary refinement. We used the training performance of each expert model as feedback to iteratively refine partition boundaries among subgraphs. These partitions are distinct for each model and graph network. We provide theoretical proof of convergence for DDGP boundary refinement as a guarantee for model training stability. Finally, we demonstrate experimentally that DDGP outperforms state-of-the-art graph partitioning algorithms for a regression task on multiple large real world graphs, with GraphSage and Graph Attention as our expert models.