Integrating Functional and Structural Connectivities via Diffusion-Convolution-Bilinear Neural Network

Abstract

Traditional brain network methods usually focus on either functional connectivity (FC) or structural connectivity (SC) for describing node interactions and only consider the interaction between paired network nodes. Therefore, the underlying relationship between FC and SC, as well as the complicated interactions among network nodes, has not been sufficiently studied and fully utilized to discover disease-related biomarkers. To tackle these problems, we propose a Diffusion-Convolution-Bilinear Neural Network (DCB-NN) framework for brain network analysis, which couples FC and SC seamlessly and considers wider interactions among network nodes. Specifically, a brain network model (graph) is first defined, whose edges are determined by neural fiber physical connections extracted from DTI and node features are governed by brain activities extracted from fMRI. Then, based on this model, we build two DCB modules to extract multi-scale features from this brain network. Each DCB module consists of diffusion, convolution and bilinear pooling. Through diffusion guided by physical connections, the network node features not only reflect the activities in their corresponding brain regions, but also are influenced by the activities from other brain regions. These enhanced node features are nonlinearly weighed through 1-D convolution, and their second-order statistics are further extracted by bilinear pooling for disease prediction. In order to capture node interactions at multi-scale, we include two DCB modules, corresponding to one-step and two-step diffusions, respectively. The whole model is trained in an end-to-end way. Experiments on a real epilepsy dataset demonstrate the effectiveness and advantages of our proposed method.