Multi-Perspective Knowledge Distillation of LLM for NER in IPE Courses

Abstract

Named Entity Recognition (NER) is essential for extracting meaningful entities from text, but existing methods struggle with complex linguistic structures and domain-specific contexts, such as those in Ideological and Political Education (IPE) texts. This paper proposes a novel approach using multi-perspective knowledge distillation from Large Language Models (LLMs) to enhance NER performance in IPE. The method involves constructing a specialized dataset for IPE and generating intermediate reasoning data using the Qwen14B model through a Chain-of-Thought (CoT) approach. Knowledge from the LLM is then distilled into a smaller NER model using techniques like DoRA fine-tuning and multi-perspective alignment, which includes feature, content, and distribution alignment. Experiments show significant improvements over state-of-the-art models, with gains of 3.46% in precision, 5.79% in recall, and 2.54% in F1 score. The method also demonstrates strong few-shot learning capabilities, achieving high performance with limited training data.