Toward Conversational Decision Support Systems: Integrating LLMs in the Operations Research Methodology

Abstract

This paper introduces the concept of Conversational Decision Support Systems (C-DSS) - a novel, agent-based framework that leverages Large Language Models (LLMs) to enhance the Operations Research (OR) methodology. We focus on the modeling and coding stages of decision support systems, where language-based interaction is crucial. The paper evaluates the effectiveness of LLMs in generating mathematical models and AMPL code for a curated set of 20 LP/MILP artifacts. Four architectural setups are analyzed: a monolithic LLM agent (M/C), its enhancement with a code verifier (M/C+V), agent-based decomposition with RAG-enhanced coding (M+CR+V), and full specialization with RAG-enhanced modeling and coding (MR+CR+V). Experimental results on two benchmark problems reveal that the targeted retrieval-augmented generation technique (RAG) significantly improves performance for complex modeling patterns such as piecewise functions, indicator constraints, and nested logic. We also propose a broader vision of C-DSS as a multi-agent ecosystem - including agents for visualization, explanation, verification, and orchestration - suggesting a path toward more explainable, adaptable, and intelligent decision support systems.