A generalization of recurrent neural networks for graph embedding

Abstract

Due to the ubiquity of graphs, machine learning on graphs facilitates many AI systems. In order to incorporate the rich information of graphs into machine learning models, graph embedding has been developed, which seeks to preserve the graphs into low dimensional embeddings. Recently, researchers try to conduct graph embedding via generalizing neural networks on graphs. However, most existing approaches focus on node embedding, ignoring the heterogeneity of edges. Besides, the similarity relationship among random walk sequences has been rarely discussed. In this paper, we propose a generalization of Recurrent Neural Networks on Graphs (G-RNN) for graph embedding. More specifically, first we propose to utilize edge embedding and node embedding jointly to preserve graphs, which is of great significance in multi-relational graphs with heterogeneous edges. Then we propose the definition of subgraph level high-order proximity to preserve the inter-sequence proximity into the embeddings. To verify the generalization of G-RNN, we apply it to the embedding of knowledge graph, a typical multi-relational graph. Empirically we evaluate the resulting embeddings on the tasks of link prediction and node classification. The results show that the embeddings learned by G-RNN are powerful on both tasks, producing better performance than the baselines.