Enhancing Performance Models with Intelligent Configuration Space Coverage

Abstract

Accurate modeling of system performance as a function of various configuration variables (CVs) requires performance measurement for a diverse set of configurations. However, the available data is often limited and covers only a small portion of the configuration space, making it insufficient for robust modeling. Collecting additional data is both expensive and challenging, particularly in production environments where measurements are time-consuming and may inconvenience users. In this paper, we introduce an Intelligent Configuration Space Coverage (ICSC) methodology that identifies the regions of the configuration space where additional performance measurements would be most beneficial for accurate modeling, while explicitly limiting the number of such measurements required. We demonstrate that our methodology substantially enhances the accuracy of performance predictions compared to methods that choose the data points randomly or via simple considerations of gaps in the CV values. Furthermore, we show that the methodology is highly valuable even for semi-supervised learning scenarios where no new measurement campaign is needed.