A high-performance processing system for monitoring stock market data stream

Abstract

High-performance real-time monitoring of the stock market data stream is one of the challenging use cases of stream processing systems. By monitoring real-time stock price patterns via high-throughput event stream processing, real-time stock trading advice can be generated. In this paper, we describe our implementation of a system for the DEBS 2022 Grand Challenge to extract breakout patterns from real-time stock market data. Breakout patterns, based on exponential moving averages crossover events, indicate potential bullish or bearish trends, giving investors insight and the opportunity to buy or sell at optimal times. We report details of our high-performance implementation to extract such patterns from a real-time stream to generate stock trading advice event notification outputs. Furthermore, we report the architectural design of our system for parallel processing of data stream, which we implemented from scratch using Python and Java programming languages.