Dynamic partition of large graphs combining local nodes exchange with directed dynamic maintenance

Abstract

Graph partition is the key preprocessing of query and analysis for graphs. In the era of big data, graphs have the characteristics of large scale and dynamic evolution. For such large dynamic graphs, the existing graph partition methods have the problems of too slow partition speed, unable to realize dynamic update, and uneven load caused by dynamic changes of graphs. Regarding to the above problems, in this paper, a processing technique combining initial graph partition with incremental dynamic maintenance is proposed. In the initial partition stage, a multi-level local nodes exchange partition algorithm is proposed, which is composed of graph compression, local node exchange partition and restoration optimization. Then an optimization adjustment mechanism is proposed to eliminate redundant modification and reduce computing cost. In the dynamic maintenance stage, several update strategies for different changes are executed. And a directed dynamic maintenance strategy is proposed to avoid frequent or circular exchange caused by two-way movement, so as to improve the efficiency of dynamic graph partition. The experiments show that our proposed method is quite efficient in dynamic partition of large graphs, which is performed both on real and synthetic data.