Key Clarity is Blue, Relaxed, and Maluma: Machine Learning Used to Discover Cross-Modal Connections Between Sensory Items and the Music They Spontaneously Evoke

Abstract

Semantic differential is often used to investigate the relationship between music and other sensory modalities such as colors, tastes, vision, and odors. This work proposes an exploratory approach including open-ended responses and subsequent machine learning to study cross-modal associations, based on a recently developed sensory scale that does not use any explicit verbal description. Twenty-five participants were asked to report a piece of music they considered close to the feel/look/experience of a given sensory stimulus. Results show that the associations reported by the participants can be explained, at least in part, by a set of features related to some timbric and tonal aspects of music.