Towards semi-automated parallelization of data stream processing

Abstract

Current hardware development trends exhibit clear inclination towards parallelism. Multicore CPUs as well as many-core architectures such as GPUs or Xeon Phi devices are widely present in both high-end servers and common desktop PCs. In order to utilize the computational power of these parallel platforms, the applications must be designed in a way that intensively exploits parallel processing. In ourwork,we propose techniques that simplify the application decomposition process in data streaming systems. The data streaming paradigm may be applied in many data-intensive applications, e.g., database management systems or scientific data processing. In order to employ these techniques, we have developed a data streaming language called Bobolang that simplifies the design of the application. This approach allows the programmer to write strictly serial operators in a traditional language and then interconnect these operators in an execution plan, that presents opportunities for automated parallel processing.