Developing natural language processing (NLP) systems for social media analysis remains an important topic in artificial intelligence research. This article introduces RoBERTweet, the first Transformer architecture trained on Romanian tweets. Our RoBERTweet comes in two versions, following the base and large architectures of BERT. The corpus used for pre-training the models represents a novelty for the Romanian NLP community and consists of all tweets collected from 2008 to 2022. Experiments show that RoBERTweet models outperform the previous general-domain Romanian and multilingual language models on three NLP tasks with tweet inputs: emotion detection, sexist language identification, and named entity recognition. We make our models (https://huggingface.co/Iulian277/ro-bert-tweet ) and the newly created corpus (https://huggingface.co/datasets/Iulian277/romanian-tweets ) of Romanian tweets freely available.
CITATION STYLE
Tăiatu, I. M., Avram, A. M., Cercel, D. C., & Pop, F. (2023). RoBERTweet: A BERT Language Model for Romanian Tweets. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 13913 LNCS, pp. 577–587). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-35320-8_44
Mendeley helps you to discover research relevant for your work.