Phase transitions and optimal algorithms for semisupervised classifications on graphs: From belief propagation to graph convolution network

Abstract

We perform theoretical and algorithmic studies for the problem of clustering and semisupervised classification on graphs with both pairwise relational information and single-point attribute information, upon a joint stochastic block model for synthetic graphs with both item-item edges and item-attribute edges. Asymptotically exact analysis based on the Bayesian inference of the model are conducted, using the cavity method in statistical physics. Analytically, we identify a phase transition of the generative model, which poses fundamental limits on the detectability of the underlying model in the clustering task for all possible algorithms. Algorithmically, we propose a belief propagation algorithm that is asymptotically optimal on the generative model, which can be further extended to a belief propagation graph convolution neural network (BPGCN) for semisupervised classification on graphs. Well-controlled benchmark data sets of factor graphs accompanied with asymptotically optimal solutions in classification could be produced for the evaluation of graph convolution neural networks and for the theoretical understanding of their strengths and weaknesses. In particular, on these synthetic benchmark networks we observe that existing graph convolution neural networks are subject to an sparsity issue and an overfitting issue in practice, both of which could be successfully overcome by our BPGCN. Moreover, when combined with classic neural network methods, BPGCN yields extraordinary classification performances on real-world data sets that are at least comparable to state-of-the-art graph convolution networks.