Enhancing large language models: alleviating knowledge deficiency with external knowledge and semantically aware reasoning (SAR)

Abstract

Inspired by the study of the human thought process, which is categorized into two systems reflecting the brain’s balancing act between speed and cognition, we propose a dual-process architecture that augments System 1 with additional knowledge akin to System 2 in human cognition. The methodology is demonstrated through FAQ retrieval task, showcasing the potential for human-like cognitive processing. Challenged by current data-driven large language models (LLMs) in reasoning and knowledge depth, this work presents a novel approach to improving conversational understanding. We leverage advanced text analysis to strategically extract key information from FAQs and utilize LLM-generated questions combined with robust semantic similarity metrics to significantly improve the precision of user query matching. The results indicate better semantic understanding and reasoning, offering a promising pathway to advancing LLM capabilities in conversational contexts. The base LLM (SBERT) enhanced with semantic textual similarity using Sentence-BERT (STS-SBERT) achieves a mean Average Precision (mAP) of 0.6165, compared to 0.3600 for SBERT alone. By strategically integrating key sentence extraction during knowledge preparation, generating questions, and applying semantic textual similarity measures, our model achieves a substantial improvement in user query matching precision. However, the activation of semantically aware reasoning (SAR) remains an issue for future research.