Sensory Perception Is a Holistic Inference Process

Abstract

Sensory perception is widely considered an inference process that reflects the best guess of a stimulus feature based on uncertain sensory information. Here we challenge this reductionist view and propose that perception is rather a holistic inference process that operates not only at the feature but jointly across all levels of the representational hierarchy. We test this hypothesis in the context of a commonly used psychophysical matching task in which subjects are asked to report their perceived orientation of a test stimulus by adjusting a probe stimulus (method-of-adjustment). We introduce a holistic matching model that assumes that subjects’ reports reflect an optimal match between the test and probe stimulus, both in terms of their inferred feature (orientation) and also their higher level representation (orientation category). Validation against several existing data sets demonstrates that the model accurately and comprehensively predicts subjects’ response behavior and outperforms previous models both qualitatively and quantitatively. Moreover, the model generalizes to other feature domains and offers an alternative account for categorical color perception. Our results suggest that categorical effects in sensory perception are ubiquitous and can be parsimoniously explained as optimal behavior based on holistic sensory representations.