Nonequilibrium Behavior in Neural Networks: Criticality and Optimal Performance

Abstract

We present a general theory which allows one to study the effects on emergent, cooperative behavior of a complex interplay between different dynamic processes that occur in actual systems at the neuron, synapse and network levels. We consider synaptic changes at different time scales from less than the millisecond to the scale of learning, and the possibility of finding a fraction of silent neurons. For some limits of interest, the fixed-point solutions or memories then loose stability and the system shows enhancement of its response to changing external stimuli for particular network topologies and dynamical memories. We observe at the edge of chaos that the network activity becomes critical in the sense that the relevant quantities show non-trivial, power–law distributions. We also describe the effect of activity–dependent synaptic processes on the network storage capacity.