Dynamic Parallel Merge Sort Method for Big Data in Multiple Isomorphic Subsystems

Abstract

Large organizations often deploy isomorphic business subsystems across regions to facilitate uniform business control and expansion. These independent subsystems, which share identical data structures, generate vast amounts of data. However, efficiently merging data from the subsystems to achieve company-wide data analysis poses a significant challenge. Therefore, this study proposes a parallel dynamic two-way merge sort algorithm. It enables rapid aggregation and sorting of big isomorphic data extracted from various regions, facilitating real-time big data analytics applications. This algorithm integrates a dynamic merging strategy to reduce data waiting time. It employs parallel merging methods to enhance merging speed. Additionally, it utilizes a custom insertion sort algorithm for the ordered results post-merging. To validate the effectiveness of this algorithm, simulation experiments on the emulated big data of the State Grid's marketing have been conducted and demonstrate that this algorithm is rapid and effective. Compared to simple merge sort, it improves efficiency by 50%. These findings suggest that the algorithm provides crucial insights for rapidly merging and sorting distributed big data.