A comparative study of text classification and missing word prediction using bert and ulmfit

Abstract

We perform a comparative study on the two types of emerging NLP models, ULMFiT and BERT. To gain insights on the suitability of these models to industry-relevant tasks, we use Text classification and Missing word prediction and emphasize how these two tasks can cover most of the prime industry use cases. We systematically frame the performance of the above two models by using selective metrics and train them with various configurations and inputs. This paper is intended to assist the industry researchers on the pros and cons of fine-tuning the industry data with these two pre-trained language models for obtaining the best possible state-of-the-art results.