Automated machine learning for test case prioritisation

Abstract

Test case prioritisation (TCP) involves ordering and selecting the most relevant test cases to verify that the current functionality of a software system remains unaffected by code changes. Recently, TCP has been addressed by machine learning (ML), predicting the failure probability of each test case. However, software engineers may struggle to identify and implement the most suitable predictive models for TCP. As new builds adapt the test suite being tested, the model performance may decline with the incorporation of these new builds. In this study, we address these challenges by applying automated workflow composition, including algorithm selection and hyperparameter optimisation. They are considered tasks within automated machine learning (AutoML). With this aim, our proposal employs grammar-guided genetic programming as the underlying mechanism for implementing the AutoML algorithm. Our experimental results demonstrate that our approach can adapt to the particularities of the system under test, selecting the most appropriate ML pipeline and hyperparameters for each build. More importantly, our approach reduces the ML knowledge required by testers—particularly regarding the selection and tuning of algorithms—while enabling them to generate pipelines suited to successive changes in SUT builds. This research showcases the potential of AutoML in software engineering, specifically for the TCP problem.