Keyword-RAG: Enhancing precision in question-answering for FDM troubleshooting from technical documents

Abstract

With accelerating digital transformation in manufacturing, large language models (LLMs) are increasingly being considered for leveraging domain-specific knowledge for decision-making and problem-solving. However, LLMs are prone to exhibiting hallucination when dealing with external information or specialized domain knowledge that falls beyond their training scope. To mitigate this issue, a retrieval-augmented generation (RAG) approach that incorporates external documents is used. However, conventional RAG approaches rely on document chunking and similarity-based retrieval, which often result in semantic fragmentation and information loss when applied to domain documents with extended contexts, thereby reducing retrieval accuracy. To overcome this problem, this study proposes Keyword-RAG, a novel RAG-based method that integrates domain-specific keyword extraction and insertion to preserve semantic coherence. Experimental evaluations using a 3D printing troubleshooting guide demonstrated that Keyword-RAG achieved superior performance, with a context recall of 0.728 and a context precision of 0.599, significantly outperforming Naive-RAG and Rerank-RAG. A qualitative evaluation further confirmed its superiority in identifying complex causal relationships and providing technically accurate solutions for 3D printing defects. These results indicate that the proposed method improves response quality in unstructured technical document environments. These improvements are particularly relevant to small and medium-sized manufacturing enterprises, where accurate, context-rich retrieval can enhance decision-making reliability and reduce troubleshooting time. Furthermore, the proposed approach can be deployed in standard on-premises computing environments, supporting practical use in real-world manufacturing settings.