Learning physical interactions to compose biological large language models

Abstract

Deep learning models have become fundamental tools in drug design. In particular, large language models trained on biochemical sequences learn feature vectors that guide drug discovery through virtual screening. However, such models do not capture the molecular interactions important for binding affinity and specificity. Therefore, there is a need to merge representations from distinct biological modalities to effectively represent molecular complexes. We present an overview of the methods to combine molecular representations and propose that future work should develop biochemical foundation models that jointly encode diverse molecular modalities. Specifically, learning to merge the representations from internal layers of domain specific biological language models could improve generalizability in the context of interaction prediction. We demonstrate that ‘composing’ biochemical language models performs similar or better than standard methods representing molecular interactions despite having significantly fewer features. We also discuss recent methods for interpreting and democratizing large language models that could aid the development of interaction aware foundation models for biology. Finally, we present a vision for future research that allows for predicting the evolution of molecular interactions across biophysical contexts.