Genetic Programming with Embedded Feature Construction for High-Dimensional Symbolic Regression

Abstract

Feature construction is an effective way to eliminate the limitation of poor data representation in many tasks such as high-dimensional symbolic regression. Genetic Programming (GP) is a good choice for feature construction for its natural ability to explore the feature space to detect and combine important features. However, there is very little contribution devoted to enhance the generalisation performance of GP for high-dimensional symbolic regression by feature construction. This work aims to develop a new feature construction method namely genetic programming with embedded feature construction (GPEFC) for high-dimensional symbolic regression. GPEFC keeps track of new small informative building blocks on best fitness gain individuals and constructs new features using these building blocks. The new constructed features augment the Terminal Set of GP dynamically. A series of experiments were conducted to investigate the learning ability and generalisation performance of GPEFC. The results show that GPEFC can evolve more compact models in an efficient way, has better learning ability and better generalisation performance than standard GP.