Automatic management of cloud applications with use of proximal policy optimization

Abstract

Reinforcement learning is a very active field of research with many practical applications. Success in many cases is driven by combining it with Deep Learning. In this paper we present the results of our attempt to use modern advancements in this area for automated management of resources used to host distributed software. We describe the use of an autonomous agent that employs a policy trained with use of Proximal Policy Optimization algorithm. The agent is managing a cloud infrastructure used to process a sample workload. We present the design and architecture of a complete autonomous management system and explain how the management policy was trained. Finally, we compare the performance to the traditional automatic management approach exploited in AWS stack and discuss feasibility to use the presented approach in other scenarios.