Network Design and the Brain

Abstract

Neural circuits have evolved to accommodate similar information processing challenges as those faced by engineered systems. Here, we compare neural versus engineering strategies for constructing networks. During circuit development, synapses are overproduced and then pruned back over time, whereas in engineered networks, connections are initially sparse and are then added over time. We provide a computational perspective on these two different approaches, including discussion of how and why they are used, insights that one can provide the other, and areas for future joint investigation. By thinking algorithmically about the goals, constraints, and optimization principles used by neural circuits, we can develop brain-derived strategies for enhancing network design, while also stimulating experimental hypotheses about circuit development and function. Network design, both in the brain and in engineering, is a fundamental problem with many shared challenges and constraints. Neural circuits in the brain develop via synaptic pruning; a process by which connections are overproduced and then eliminated over time. In contrast, computer scientists typically design networks by starting with an initially sparse topology and gradually adding connections. We discuss possible advantages and disadvantages of these two design strategies, when different design strategies may be useful, and how the study of one can shed light on the other. We also discuss optimization principles shared by neural and engineered systems more broadly, and we propose new questions for joint investigation. The focus on extracting abstract, algorithmic principles derived from neural circuits to enhance network design represents an exciting area of synergy between neurobiologists and computer scientists.