Learning with hidden variables

Abstract

Learning and inferring features that generate sensory input are crucial information processing tasks continuously performed by cortical networks. In recent years, novel algorithms and learning rules have been proposed that allow neuronal networks to learn such features from complex datasets. These networks involve layers of hidden neurons leading to a remarkable increase in their ability to explain complex sensory inputs. Here we review recent advancements in this line of research emphasizing, amongst other things, the processing of dynamical inputs by networks with hidden nodes. We identify issues which in our view should be explored to achieve a better understanding of the relationship between machine learning approaches to learning in networks with hidden nodes and learning in cortical circuits. Keywords: statistical models, deep learning, dynamics