Processing partially ordered requests in distributed stream processing systems

Abstract

In many application scenarios of distributed stream processing, there might be partial order relations among the requests. However, existing stream processing systems can not directly handle partially ordered requests, while indirect mechanisms are usually strongly coupled with business logic, which lack flexibility and have limited performance. We propose Pork, a novel distributed stream processing system targeting at partially ordered requests. In the experiments, the new system has achieved a parallelism and request throughput larger than the traditional mechanism in the presented example, and the performance overhead due to parallelism is considerably small. Then the scalability characteristic of the new system is discussed. What’s more, the experiment results also show that the new system has a more flexible load balancing ability.