NEAT in HyperNEAT Substituted with Genetic Programming Zden�� ek Buk, Jan koutn�� ��k, and Miroslav Snorek�� Computational Intelligence Group Department of Computer Science and Engineering Faculty of Electrical Engineering Czech Technical University in Prague {bukz1,koutnij,snorek}@fel.cvut.cz http://cig.felk.cvut.cz Abstract. In this paper we present application of genetic programming (GP) [1] to evolution of indirect encoding of neural network weights. We compare usage of original HyperNEAT algorithm with our implementa- tion, in which we replaced the underlying NEAT with genetic program- ming. The algorithm was named HyperGP. The evolved neural networks were used as controllers of autonomous mobile agents (robots) in simu- lation. The agents were trained to drive with maximum average speed. This forces them to learn how to drive on roads and avoid collisions. The genetic programming lacking the NEAT complexification property shows better exploration ability and tends to generate more complex solutions in fewer generations. On the other hand, the basic genetic program- ming generates quite complex functions for weights generation. Both approaches generate neural controllers with similar abilities. 1 Introduction In training of artificial neural networks using evolutionary algorithms a method of encoding the neural networks into individuals in the evolution is needed. Basically, there are two types of encodings. Direct encoding of either connection weights or a network structure causes individuals and therefore the search space complexity growth. Indirect encoding that utilizes a system, which develops the neural network from information encoded by the individual can overcome such drawback of direct encoding. One of the most perspective algorithm of neural network weights and struc- ture encoding is the hypercube encoding invented by Ken Stanley [2,3] as Hy- perNEAT algorithm. The HyperNEAT algorithm consists of a function or a set of functions, which generates weights for neural network. The neural network consist of neurons placed in a rectangular mesh called substrate. The substrate coordinates serve as inputs for the function. The function output is the weight of connection between two neurons. In the original HyperNEAT algorithm, the function is called CPPN (Compositional Pattern Production Network) and is constructed from a set of nodes with scalar product on their inputs and non- linear function at their outputs. The network structure reminds a neural network M. Kolehmainen et al. (Eds.): ICANNGA 2009, LNCS 5495, pp. 243���252, 2009. c Springer-Verlag Berlin Heidelberg 2009

244 Z. Buk, J. koutn�� ��k, and M. Snorek�� (therefore, it is named network rather than a function). The CPPN in the Hy- perNEAT is generated using the NEAT (NeuroEvolution of Augmenting Topolo- gies) [3] algorithm. NEAT is a type of evolutionary algorithm, which evolves the network from a simple form featuring complexification and niching. The NEAT algorithm is the component that we decided to replace with a different style of weights encoding and generation in our approach. Rather, we use genetic programming, which generates functions that compute weights for connections among neurons in the substrate. The application domain is a control of autonomous agents in simulated envi- ronment. The agents are equipped with sensors with scalable resolution. Encod- ing of neurons weights allows the resolution of the sensory input to be changed independently of the size of the individual that contains the weight generating function. The previous experiments presented in [4] show that the NEAT can produce recurrent neural network, which can control the agent to move through the simulated environment with maximum average speed. The fitness of the evolved neural network is the average speed of the controlled agent. Our goal is to replace the NEAT algorithm with genetic programming and compare it to the original HyperNEAT algorithm. 1.1 Related Work Many techniques for evolution of either weights or structure of neural networks were already developed such as Analog Genetic Encoding [5,6,7], Continual Evolution Algorithm [8], GNARL [9], Evolino [10] and NeuroEvolution of Aug- menting Topologies (NEAT) [11]. The NEAT algorithm became a part of the HyperNEAT algorithm as a tool for evolution of CPPNs. HyperNEAT algorithm was already applied to control artificial agents in food gathering problem [2]. It was shown that HyperNEAT is capable of large scale networks evolution ( 8 �� 106 connections). The simulated agent was equipped with concentric sensors for food in particular directions linked with effector, which drives the agent to the direction. In our approach, the sensors are organized in polar rays with particular angular and distance resolution. The sensors are sensitive to the surface color. The agents can share portion of one substrate together [12]. The substrate splits to local but linked areas. The agents can exploit cooperative behavior afterwards. Agents can complete common goals also with a minimum information from the sensors with evolutionary trained feed forward networks as well [13]. In the case, the agents exhibit reactive behavior. This paper is organized as follows. Section 2 describes the HyperGP algorithm. Section 3 describes the simulation environment and the agent setup. Section 4 describes the experimental results, performance of HyperGP is compared with HyperNEAT. Final section concludes the paper.