HW-TSC at SemEval-2023 Task 7: Exploring the Natural Language Inference Capabilities of ChatGPT and Pre-trained Language Model for Clinical Trial

Abstract

In this paper, we describe a effective system for SemEval-2022 Task 7. This task aims to determine whether a given statement is supported by comparing one or two clinical trial reports, and to identify evidence that supports the statement. This is a task that requires high natural language inference capabilities. In Subtask 1, we compare our strategy based on prompt learning and ChatGPT with a baseline constructed using BERT in zero-shot setting, and validate the effectiveness of our strategy. In Subtask 2, we fine-tune DeBERTaV3 for classification without relying on the results from Subtask 1. We find that early stopping of the training can effectively prevent model overfitting, and this achieves a good performance in Subtask 2. In addition, we do not use any ensemble strategies. We have achieved the 10th place in Subtask 1 and the 2nd place in Subtask 2.