AMESoS: A Scalable and Elastic Framework for Latency Sensitive Streaming Pipelines

Abstract

Serverless computing, and in particular Function as a Service (FaaS), has become an increasingly popular cloud programming model in recent years. The serverless computing model offers an intuitive, event-based interface for developing cloud-based applications, that makes the writing and deployment of scalable microservices easier and cost-effective. Existing orchestrators in serverless systems are mainly designed for short-lived functions. Nevertheless, an increasing number of applications are deployed as pipelines that comprise a sequence of functions that execute in a specific order or pattern and must meet a wide range of throughput and latency targets to be practical. In this paper, we present AMESoS, our scalable and elastic framework for latency-sensitive streaming pipelines. AMESoS (i) enables developers to build predictable pipelines that meet their latency demands, (ii) employs prediction to proactively estimate the most appropriate number of active replicas needed for each function in the pipeline, and (iii) dynamically scales the number of replicas for each pipeline's functions in the presence of overloads. Our experimental results demonstrate the efficiency and benefits of our approach over state of the art systems.