Convolutional neural network with biologically inspired ON/OFF ReLU

Abstract

This paper proposes a modification of convolutional neural network (CNN) with biologically inspired structure, retinal structure and ON/OFF rectified linear unit (ON/OFF ReLU). Retinal structure enhances input images by center surround difference of green and red, blue and yellow components and creates positive results and negative results like ON/OFF visual pathway of retina to make totally 12 feature channels. This ON/OFF concept also adopted to each convolutional layer of CNN and we call this ON/OFF ReLU. Conventional ReLU only passes positive results of each convolutional layer so it loses negative information such as how much it was negative and also loses learning chance if results are saturated to zero but proposed model uses both positive and negative information so that additional learning chance also exist through negative results. Experimental results show how much the negative information and retinal structure improves the performance of CNN with public data.