Acceptance Criteria Validation in Agile Projects Using AI and NLP Techniques

Abstract

In agile software development, user stories and their acceptance criteria play a critical role in ensuring alignment between stakeholder expectations and system functionality. However, the manual validation of these criteria is often labor-intensive and prone to bias. This study investigates the application of Artificial Intelligence (AI) techniques, particularly Natural Language Processing (NLP) and Machine Learning (ML), to automate the analysis and validation of user stories. Using a dataset of user stories collected from academic and industry projects, we trained and evaluated four ML algorithms: Multilayer Perceptron (MLP), Support Vector Machine (SVM), Naive Bayes, and Random Forest. The models were assessed for their ability to classify acceptance criteria accurately and efficiently. Our findings demonstrate the potential of AI to enhance the validation process, achieving over 60% accuracy in certain cases, with SVM standing out as the most robust algorithm. This research highlights the transformative role of AI in improving software requirements analysis and lays the foundation for future innovations in automated validation and quality assurance in agile environments.