Can LLM Substitute Human Labeling? A Case Study of Fine-grained Chinese Address Entity Recognition Dataset for UAV Delivery

Abstract

We present CNER-UAV, a fine-grained Chinese Name Entity Recognition dataset specifically designed for the task of address resolution in Unmanned Aerial Vehicle delivery systems. The dataset encompasses a diverse range of five categories, enabling comprehensive training and evaluation of NER models. To construct this dataset, we sourced the data from a real-world UAV delivery system and conducted a rigorous data cleaning and desensitization process to ensure privacy and data integrity. The resulting dataset, consisting of around 12,000 annotated samples, underwent human experts and Large Language Model annotation. We evaluated classical NER models on our dataset and provided in-depth analysis. The dataset and models are publicly available at https://github.com/zhhvvv/CNER-UAV.