Do Google App Engine's Runtimes Perform Homogeneously? An Empirical Investigation for Bonus Computing

Abstract

Bonus computing is a new metacomputing form that takes advantage of free computing power from the public Cloud market. To maximize the value of free Cloud resources and facilitate dividing Bonus computing tasks, it would particularly be crucial to understand the performance of candidate Cloud services before using them in production. By offering free quotas in its standard environment, Google App Engine (GAE) has become a popular public Platform-As-A-Service (PaaS) for Bonus computing. Since GAE natively supports various programming languages with flexible configurations (e.g., region selection), it will be possible and valuable to squeeze GAE's free computing power if there is an optimal choice of its different runtimes. Following the performance evaluation methodology DoKnowMe, we implemented several versions of the Fibonacci(-like) calculation as benchmarks to fundamentally investigate GAE's standard environment. Our investigation results reveal that GAE does not support its runtime environments homogeneously in terms of their computation speed and memory efficiency. The heterogeneity could be related not only to the characteristics of different programming languages but also to the diverse GAE infrastructures. For example, Go runtime seems to be a well-Trade-off to satisfy Bonus computing among all the options, while the GAE service located in southamerica-east1 and us-central1 performs dramatically worse than that in the other regions.