Flex-GP: Genetic programming on the cloud

Abstract

We describe Flex-GP, which we believe to be the first largescale genetic programming cloud computing system. We took advantage of existing software and selected a socket-based, client-server architecture and an island-based distribution model. We developed core components required for deployment on Amazon's EC2. Scaling the system to hundreds of nodes presented several unexpected challenges and required the development of software for automatically managing deployment, reporting, and error handling. The system's performance was evaluated on two metrics, performance and speed, on a difficult symbolic regression problem. Our largest successful Flex-GP runs reached 350 nodes and taught us valuable lessons for the next phase of scaling. © 2012 Springer-Verlag.