A Baseline Symbolic Regression Algorithm

Abstract

Recent advances in symbolic regression (SR) havepromoted the field into the early stages of commercialexploitation. This is the expected maturation historyfor an academic field which is progressing rapidly. Theoriginal published symbolic regression algorithms in(Koza 1994) have long since been replaced by techniquessuch as Pareto front, age layered populationstructures, and even age Pareto front optimisation. Thelack of specific techniques for optimising embeddedreal numbers, in the original algorithms, has beenreplaced with sophisticated techniques for optimizingembedded constants. Symbolic regression is coming ofage as a technology.As the discipline of Symbolic Regression (SR) hasmatured, the first commercial SR packages haveappeared. There is at least one commercial package onthe market for several years http://www.rmltech.com/.There is now at least one well documented commercialsymbolic regression package available for Mathmaticawww.evolved-analytics.com. There is at least one verywell done open source symbolic regression packageavailable for free downloadhttp://ccsl.mae.cornell.edu/eureqa. Yet, even as thesophistication of commercial SR packages increases,there have been glaring issues with SR accuracy even onsimple problems (Korns 2011). The depth and breadth ofSR adoption in industry and academia will be greatlyaffected by the demonstrable accuracy of available SRalgorithms and tools.In this chapter we develop a complete public domainalgorithm for modern symbolic regression which isreasonably competitive with current commercial SRpackages, and calibrate its accuracy on a set ofpreviously published sample problems. This algorithm isdesigned as a baseline for further public domainresearch on SR algorithm simplicity and accuracy. Noclaim is made placing this baseline algorithm on a parwith commercial packages, especially as the commercialofferings can be expected to relentlessly improve inthe future. However this baseline is a greatimprovement over the original published algorithms, andis an attempt to consolidate the latest publishedresearch into a simplified baseline algorithm ofsimilar speed and accuracy.The baseline algorithm presented herein is called AgeWeighted Pareto Optimisation. It is an amalgamation ofrecent published techniques in Pareto frontoptimization (Kotanchek et al., 2007), age layeredpopulation structures (Hornby 2006), age fitness Paretooptimization (Schmidt and Hipson 2010), and specialisedembedded abstract constant optimization (Korns 2010).The complete pseudo code for the baseline algorithm ispresented in this paper. It is developed step by stepas enhancements to the original published SR algorithm(Koza 1992) with justifications for each enhancement.Before-after speed and accuracy comparisons are madefor each enhancement on a series of previouslypublished sample problems.