Bridging the Gap: Ensemble Learning-Based NLP Framework for AI-Generated Text Identification in Academia

Abstract

Background: The advent of Large Language Models (LLMs), including Chat Generative Pre-trained Transformer (ChatGPT) and Bard, has revolutionised text generation while raising ethical concerns regarding academic integrity. Differentiating Artificial Intelligence-Generated Texts (AIGT) from human-written content is crucial to maintaining transparency and trust in scholarly communication. Objective: This study aims to address the limitations in existing detection methods by introducing a Machine Learning (ML)-based Natural Language Processing (NLP) framework that effectively distinguishes between AI-generated and Human-Written academic texts (HWAI). Methodology: The proposed framework integrates comprehensive preprocessing, Exploratory Data Analysis (EDA), linguistic analysis, and ensemble learning techniques. Text representation was achieved using Term Frequency-Inverse Document Frequency (TF-IDF) and word embeddings. We employed two diverse datasets, Artificial Intelligence-Generated Academic (AI-GA) and HWAI, to validate the framework’s efficacy, ensuring robust classification performance. Results: The ensemble model did better than individual classifiers. On the AI-GA dataset, it achieved state-of-the-art accuracy (98.67%) and Receiver Operating Characteristic-Area Under the Curve (ROC-AUC) (99.88%). The HWAI dataset achieved 96.52% accuracy and 99.37% ROC-AUC. These results highlight the framework’s capability to identify unique linguistic patterns in AI-generated content. Conclusion: The framework addresses key linguistic and computational challenges and provides a scalable and reliable solution for detecting AI-generated content in academic domains. Future work will explore hybrid human-AI authorship detection and real-time deployment to enhance its practical utility across disciplines.