SCAD: Scalability Advisor for Interactive Microservices on Hybrid Clouds

Abstract

The microservice architecture allows scaling application components independently based on their resource demands to serve user traffic. The notion of user traffic is critical because it is a mixture of requests to user-facing API endpoints representing valuable semantics (e.g., a customer transaction). Application owners can incorporate business insights to derive the expected user traffic, e.g., for holiday seasons, and rightsize each component to ensure availability and responsiveness. However, existing resource estimation techniques do not take user traffic from application owners into consideration but only rely on historical information, which leads to inaccurate predictions. Furthermore, on-premises infrastructure lacks elasticity, and the overall demands to serve the traffic can exceed its capacity, leaving no room for components to grow. Hybrid clouds provide an attractive solution by offloading some components to the cloud. However, a poor choice to offload can worsen the application in multiple aspects. To address these problems, we introduce SCAD, a scalability advisor for resource management. It estimates resource demands for any user traffic provided by the application owner and recommends how to scale microservices by spanning them on hybrid clouds, optimizing API performance, API availability, and cloud hosting cost.