Predicting task-related brain activity from resting-state brain dynamics with fMRI Transformer

Abstract

Accurate prediction of the brain’s task reactivity from resting-state functional magnetic resonance imaging (fMRI) data remains a significant challenge in neuroscience. Traditional statistical approaches often fail to capture the complex, nonlinear spatiotemporal patterns of brain function. This study introduces SwiFUN (Swin fMRI UNet Transformer), a novel deep learning framework designed to predict 3D task activation maps directly from resting-state fMRI scans. SwiFUN leverages advanced techniques such as shifted window-based self-attention, which helps to understand complex patterns by focusing on varying parts of the data sequentially, and a contrastive learning strategy to better capture individual differences among subjects. When applied to predicting emotion-related task activation in adults (UK Biobank, n = 7,038) and children (ABCD, n = 4,944), SwiFUN consistently achieved higher overall prediction accuracy than existing methods across all contrasts; it demonstrated an improvement of up to 27% for the FACES-PLACES contrast in ABCD data. The resulting task activation maps revealed individual differences across cortical regions associated with sex, age, and depressive symptoms. This scalable, transformer-based approach potentially reduces the need for task-based fMRI in clinical settings, marking a promising direction for future neuroscience and clinical research that enhances our ability to understand and predict brain function.