Evolution of Spiking Neural Controllers for Autonomous Vision-Based Robots Dario Floreano and Claudio Mattiussi Evolutionary & Adaptive Systems, Institute of Robotics Swiss Federal Institute of Technology, CH-1015 Lausanne (EPFL) Dario.Floreano@epfl.ch, Claudio.Mattiussi@epfl.ch Abstract. We describe a set of preliminary experiments to evolve spik- ing neural controllers for a vision-based mobile robot. All the evolution- ary experiments are carried out on physical robots without human inter- vention. After discussing how to implement and interface these neurons with a physical robot, we show that evolution finds relatively quickly functional spiking controllers capable of navigating in irregularly tex- tured environments without hitting obstacles using a very simple genetic encoding and fitness function. Neuroethological analysis of the network activity let us understand the functioning of evolved controllers and tell the relative importance of single neurons independently of their observed firing rate. Finally, a number of systematic lesion experiments indicate that evolved spiking controllers are very robust to synaptic strength de- cay that typically occurs in hardware implementations of spiking circuits. 1 Spiking Neural Circuits The great majority of biological neurons communicate by sending pulses along the axons to other neurons. A pulse is a small current charge that occurs when the voltage potential across the membrane of a neuron exceeds its threshold. The pulse is also known as ���spike��� to indicate its short and transient nature. After emitting a spike, a neuron needs some time to reestablish its electrochem- ical equilibrium and therefore cannot immediately emit a new spike, no matter how strong its excitatory input is. A typical neuron in the cortex ���fires��� ap- proximately 10 spikes per second during resting conditions and can emit up to 300 spikes per second in operating conditions. Other neurons can fire more fre- quently (for example 500 spikes per seconds) clustered in short periods of time (���bursting neurons���). In the field of artificial neural networks we find two different classes of models that differ with respect to the interpretation of the role of spikes. Connectionist models of neural networks [19], by far the most widespread models, assume that what matters in the communication among neurons is the firing rate of a neuron. The firing rate is the average quantity of spikes emitted by the neuron over a rela- tively long time window (for example, over 100 ms). This quantity is represented by the activation level of the neuron. For example, a neuron characterized by a sigmoid activation function, such as the logistic function f(x) = 1/1 + exp(���x), T. Gomi (Ed.): ER 2001, LNCS 2217, pp. 38���61, 2001. c Springer-Verlag Berlin Heidelberg 2001

Evolution of Spiking Neural Controllers 39 that gives an output of 0.5 would be equivalent to a spiking neuron that emits approximately half its maximum rate of spikes (150 out of 300 spikes per second, e.g.). Models of pulsed neural networks [14] instead assume that the firing time, that is the precise time of emission of a single spike, can transmit important information for the post-synaptic neuron [23]. Therefore, these models use more complex activation functions that simulate the emission and reception of spikes on a very fine timescale. Spiking circuits have at least two properties that make them interesting can- didates for adaptive control of autonomous behavioral robots: ��� The intrinsic time-dependent dynamics of neuron activation could detect and exploit more easily (e.g., with simpler circuits or with higher reliabil- ity) temporal patterns of sensory-motor events than connectionist neural networks. ��� Since the physics of circuits of sub-threshold transistors (i.e., characterized by gate-to-source voltage differences below their threshold voltage) imple- mented with analog Very Large Scale Integration technology [15] match the properties of spiking neurons, it possible to implement large networks of spiking neurons in tiny and low-power chips [9]. Designing circuits of spiking neurons with a given functionality is still a challenging task and the most successful results obtained so far have focused on the first stages of sensory processing and on simple motor control. For example, Indiveri et al. [8] have developed neuromorphic vision circuits that emulate the interconnections among the neurons in the early layers of an artificial retina in order to extract motion information and a simple form of attentive selection of visual stimuli. These vision circuits have been interfaced with a Koala robot and their output has been used to drive the wheels of the robot in order to follow lines [10]. In another line of work, Lewis et al. have developed an analog VLSI circuit with four spiking neurons capable of controlling a robotic leg and adapting the motor commands using sensory feedback [13]. This neuromorphic circuit consumes less than 1 microwatt of power and takes less than 0.4 square millimeters of chip area. Despite these interesting implementations, there are not yet methods for developing complex spiking circuits that could display minimally-cognitive func- tions or learn behavioral abilities through autonomous interaction with the envi- ronment. Furthermore, the potentially complex dynamics that a spiking circuit with feedback loops can display allows several alternative functioning modali- ties. For example, a spiking circuit could use inhibitory neurons to modulate the overall excitation of the network and/or selectively interact with other neurons to inhibit specific actions. Also, the same circuit could use firing rate instead of firing time as the preferred functioning modality (this concept will be discussed in more detail later on in this paper). Artificial evolution is therefore an interesting method to discover spiking cir- cuits that autonomously develop desired behavioral abilities for robots without imposing constraints on their architecture and functioning modality. In this pa- per, we describe some initial explorations in the evolution of spiking circuits for

40 D. Floreano and C. Mattiussi Fig. 1. The function describing the contribution of a spike from a presynaptic neuron emitted at time tf . The contribution of the spike begins after some delay ��� (2 ms) due to the traveling time of the spike and eventually decreases its effect as time t flows from the firing time tf . The synapse time constant ��s is set to 10 ms and the membrane time constant ��m is set to 4 ms. a task of vision-based navigation using a Khepera robot with a linear CMOS vision system. We will then analyze the evolved spiking circuit and discuss the most important lessons learned from these experiments. Finally, we will describe some ideas for future developments in this emerging area of research. 2 The Spike Response Model The state of a spiking neuron is described by the voltage difference across its membrane, also known as membrane potential ��. Incoming spikes can increase or decrease the membrane potential. The neuron emits a spike when the total amount of excitation induced by incoming excitatory and inhibitory spikes ex- ceeds its firing threshold ��. After firing, the membrane potential of the neuron resets its state to a low negative voltage during which it cannot emit a new spike, and gradually returns to its resting potential. This recharging period is called the refractory period. There are several models of spiking neurons that account for these properties with various degrees of detail. In the experiments described in this paper, we have chosen the Spike Response Model developed by Gerstner [5]. It has been shown that several other models of spiking neurons, such as the class of Integrate-and- Fire neurons (where the membrane potential of the neuron is immediately reset to its resting value after a spike), represent special cases of the Spike Response Model [6]. In this model, the effect of an incoming spike on the neuron membrane is a function of the difference s = t ��� tf between current time t and the time when the spike was emitted tf . The properties of the function are determined by a) the delay ��� between the generation of a spike at the pre-synaptic neuron