MIReAD: simple method for learning high-quality representations from scientific documents

0Citations
Citations of this article
12Readers
Mendeley users who have this article in their library.

Abstract

Learning semantically meaningful representations from scientific documents can facilitate academic literature search and improve performance of recommendation systems. Pretrained language models have been shown to learn rich textual representations, yet they cannot provide powerful document-level representations for scientific articles. We propose MIREAD, a simple method that learns high-quality representations of scientific papers by fine-tuning transformer model to predict the target journal class based on the abstract. We train MIREAD on more than 500,000 PubMed and arXiv abstracts across over 2,000 journal classes. We show that MIREAD produces representations that can be used for similar papers retrieval, topic categorization and literature search. Our proposed approach outperforms six existing models for representation learning on scientific documents across four evaluation standards.

Cite

CITATION STYLE

APA

Razdaibiedina, A., & Brechalov, A. (2023). MIReAD: simple method for learning high-quality representations from scientific documents. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (Vol. 2, pp. 530–539). Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/2023.acl-short.46

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Save time finding and organizing research with Mendeley

Sign up for free