Foundation models for bioinformatics

Abstract

Transformer-based foundation models such as ChatGPTs have revolutionized our daily life and affected many fields including bioinformatics. In this perspective, we first discuss about the direct application of textual foundation models on bioinformatics tasks, focusing on how to make the most out of canonical large language models and mitigate their inherent flaws. Meanwhile, we go through the transformer-based, bioinformatics-tailored foundation models for both sequence and non-sequence data. In particular, we envision the further development directions as well as challenges for bioinformatics foundation models.