Elastic pulsar functions for distributed stream processing

1Citations
Citations of this article
6Readers
Mendeley users who have this article in their library.
Get full text

Abstract

An increasing number of data-driven applications rely on the ability of processing data flows in a timely manner, exploiting for this purpose Data Stream Processing (DSP) systems. Elasticity is an essential feature for DSP systems, as workload variability calls for automatic scaling of the application processing capacity, to avoid both overload and resource wastage. In this work, we implement auto-scaling in Pulsar Functions, a function-based streaming framework built on top of Apache Pulsar. The latter is is a distributed publish-subscribe messaging platform that natively supports serverless functions. Considering various state-of-the-art policies, we show that the proposed solution is able to scale application parallelism with minimal overhead.

Cite

CITATION STYLE

APA

Russo, G. R., Schiazza, A., & Cardellini, V. (2021). Elastic pulsar functions for distributed stream processing. In ICPE 2021 - Companion of the ACM/SPEC International Conference on Performance Engineering (pp. 9–16). Association for Computing Machinery, Inc. https://doi.org/10.1145/3447545.3451901

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Save time finding and organizing research with Mendeley

Sign up for free