Commonsense Reasoning and Explainable Artificial Intelligence Using Large Language Models

Abstract

Commonsense reasoning is a difficult task for a computer, but a critical skill for an artificial intelligence (AI). It can enhance the explainability of AI models by enabling them to provide intuitive and human-like explanations for their decisions. This is necessary in many areas but especially in the field of question answering (QA), which is one of the most important tasks of natural language processing (NLP). Over time, a multitude of methods have emerged for solving commonsense reasoning problems such as knowledge-based approaches using formal logic or linguistic analysis. In this paper, we investigate the effectiveness of large language models (LLMs) on different QA tasks with focus on their abilities on reasoning and producing explanations. For this, we study the recent and very prominent LLM ChatGPT and evaluate the results by means of a questionnaire. We demonstrate ChatGPT’s ability to reason with common sense, and although ChatGPT’s accuracy ranges from 56% to 93% on various QA benchmarks, it outperforms human accuracy. Furthermore we can appraise that, in the sense of explainable artificial intelligence (XAI), ChatGPT gives good explanations for its decisions. In our questionnaire we found that 68% of the participants quantify ChatGPT’s explanations as “good” or “excellent”. Taken together, these findings enrich our understanding of current LLMs and pave the way for future investigations of reasoning and explainability.