Leveraging Retrieval-augmented LLMs for Automated Test Case Generation from Software Requirements Specification

Abstract

Automating test case generation from Software Requirements Specifications (SRS) holds significant potential for improving software quality and reducing the workload on quality assurance teams. This paper presents a novel tool that leverages Retrieval-Augmented Generation (RAG) and Few-Shot prompting techniques to generate accurate and domain-adaptive test cases from natural language SRS documents. Our system combines large language models (LLMs), both proprietary (GPT-4o) and open-source (mistral:7b, phi4:14b, deepseek-r1:70b, etc.), with a semantic search engine based on sentence embeddings (MiniLM) to retrieve relevant SRS segments. These are dynamically combined with a curated few-shot examples from eight business sectors to guide the generation process. The entire pipeline is deployed in a user-friendly Streamlit interface, enabling interactive, on-demand test case generation per software module. We evaluate the system using both quantitative metrics (BLEU score, latency) and qualitative human expert reviews across six models. Results show that our full system significantly outperforms baseline prompting, achieving a BLEU score of 0.72 and a human-rated accuracy of 4.8/5. An ablation study confirms the critical role of the few-shot examples in achieving high output quality. Our findings demonstrate that even general- purpose LLMs, when properly guided, can serve as reliable tools for intelligent domain-aware software testing, offering a scalable solution for modern QA workflows.