In silico discovery of representational relationships across visual cortex

Abstract

Human vision is mediated by a complex interconnected network of cortical brain areas that jointly represent visual information. Although these areas are increasingly understood in isolation, their representational relationships remain unclear. Here we developed relational neural control and used it to investigate the representational relationships for univariate and multivariate functional magnetic resonance imaging (fMRI) responses of areas across the visual cortex. Through relational neural control, we generated and explored in silico fMRI responses for large numbers of images, discovering controlling images that align or disentangle responses across areas, thus indicating their shared or unique representational content. This revealed a typical network-level configuration of representational relationships in which shared or unique representational content varied on the basis of cortical distance, categorical selectivity and position within the visual hierarchy. Closing the empirical cycle, we validated the in silico discoveries on in vivo fMRI responses from independent participants. Together, this reveals how visual areas jointly represent the world as an interconnected network.