BYteam at SemEval-2020 Task 5: Detecting Counterfactual Statements with BERT and Ensembles

Abstract

We participate in the classification tasks of SemEval-2020 Task: Subtask1: Detecting counterfactual statements of semeval-2020 task5(Detecting Counterfactuals). This paper examines different approaches and models towards detecting counterfactual statements classification. We choose the Bert model. However, the output of Bert is not a good summary of semantic information, so in order to obtain more abundant semantic information features, we modify the upper layer structure of Bert. Finally, our system achieves an accuracy of 88.90 % and F1 score of 86.30 % by hard voting, which ranks 6th on the final leader board of the in subtask 1 competition.