Instance space analysis for a personnel scheduling problem

Abstract

This paper considers the Rotating Workforce Scheduling Problem, and shows how the strengths and weaknesses of various solution methods can be understood by the in-depth evaluation offered by a recently developed methodology known as Instance Space Analysis. We first present a set of features aiming to describe hardness of test instances. We create a new, more diverse set of instances based on an initial instance space analysis that reveals gaps in the instance space, and offers the opportunity to generate additional instances to add diversity to the test suite. The results of three algorithms on our extended instance set reveal insights based on this visual methodology. We observe different regions of strength and weakness in the instance space for each algorithm, as well as a phase transition from feasible to infeasible instances with more challenging instances at the phase transition boundary. This rigorous and insightful approach to analyzing algorithm performance highlights the critical role played by the choice of test instances, and the importance of ensuring diversity and unbiasedness of test instances to support valid conclusions.