Charon: A Framework for Microservice Overload Control

Abstract

Overload control is an important feature of modern cloud applications. As these applications grow increasingly complex, designing efficient overload control schemes at scale is tedious. In this paper we argue part of the challenge is a lack of first principles mechanisms one can use to design scalable and verifiable policies. We present Charon, a market-based scheme for large scale service graphs. Unsurprisingly, Charon relies on tokens to negotiate the acquisition of compute resources. However, unlike existing receiver-driven systems, Charon decouples the mechanisms used to generate and value tokens and proposes efficient price propagation mechanisms. We motivate Charon with a set of representative system conditions that existing frameworks cannot handle well, detail an example policy one can build using the proposed mechanisms, and discuss open research challenges our framework exposes.