Finding heterogeneous-unsafe configuration parameters in cloud systems

Abstract

With the increasing prevalence of heterogeneous hardware and the increasing need for online reconfiguration, there is increasing demand for heterogeneous configurations. However, allowing different nodes to have different configurations may cause errors when these nodes communicate, even if the configuration of each node uses valid values. To test which configuration parameters are unsafe when configured in a heterogeneous manner, this work reuses existing unit tests but runs them with heterogeneous configurations. To address the challenge that unit tests often share the configuration across different nodes, we incorporate several heuristics to accurately map configuration objects to nodes. To address the challenge that there are too many tests to run, we (1) "pre-run"unit tests to determine effective unit tests for each configuration parameter and (2) introduce pooled testing to test several parameters together. Our evaluation finds 41 heterogeneous-unsafe configuration parameters in Flink, HBase, HDFS, MapReduce, and YARN. We further propose suggestions and workarounds to make a subset of these parameters heterogeneous safe.