CloudPT: Performance testing for identifying and detecting bottlenecks in IaaS

Abstract

This work addresses performance testing for monitoring mass quantities of large-dataset measurements in infrastructure-as-a-Service (IaaS). Physical resources are not virtualized in sharing dynamic clouds; thus, shared resources compete for access to system resources. This competition introduces significant new challenges when assessing the performance of IaaS. A bottleneck may occur if one system resource is critical to IaaS; this may shut down the system and services, which would reduce the workflow performance by a large margin. To protect against bottlenecks, we propose CloudPT, a performance test management framework for IaaS. CloudPT has many advantages: (I) high-efficiency detection; (II) a unified end-to-end feedback loop to collaborate with cloud-ecosystems management; and (III) a troubleshooting performance test. This paper shows that CloudPT efficiently identifies and detects bottlenecks with a minimal false-positive rate (<13%) and it correlates high accuracy using the failure of a host virtual machine (host VM) to start-up with both cloud illustrative batches and transactional workloads such as the Spark, and Kafka framework for a data partitioning and collecting events on an each server. In a framework based on a trace case study, CloudPT diagnosed performance bottlenecks in 20 s with a precision rate of 86%, confirming its real-time efficiency.