Better Solutions Faster: Soft Evolution of Robust Regression Models InParetogeneticprogramming

Abstract

Better solutions faster is the reality of theindustrial modelling world, now more than ever.Efficiency requirements, market pressures, and everchanging data force us to use symbolic regression viagenetic programming (GP) in a highly automated fashion.This is why we want our GP system to produce simplesolutions of the highest possible quality with thelowest computational effort, and a high consistency inthe results of independent GP runs. In this chapter, weshow that genetic programming with a focus on rankingin combination with goal softening is a very powerfulway to improve the efficiency and effectiveness of theevolutionary search. Our strategy consists of partialfitness evaluations of individuals on random subsets ofthe original data set, with a gradual increase in thesubset size in consecutive generations. From a seriesof experiments performed on three test problems, weobserved that those evolutions that started from thesmallest subset sizes (10percent) consistently led toresults that are superior in terms of the goodness offit, consistency between independent runs, andcomputational effort. Our experience indicates thatsolutions obtained using this approach are also lesscomplex and more robust against over-fitting. We findthat the near-optimal strategy of allocatingcomputational budget over a GP run is to evenlydistribute it over all generations. This implies thatinitially, more individuals can be evaluated usingsmall subset sizes, promoting better exploration.Exploitation becomes more important towards the end ofthe run, when all individuals are evaluated using thefull data set with correspondingly smaller populationsizes.