LA-RCS: LLM-agent-based Robot Control System

Abstract

The large language model (LLM)-agent-based robot control system (LA-RCS) is a sophisticated robot control system designed to autonomously plan, work, and analyze the external environment on the basis of user requirements by utilizing LLM agents. Utilizing a dual-agent framework, LA-RCS generates plans on the basis of user requests, observes the external environment, executes the plans, and modifies the plans as needed to adapt to changes in external conditions. Additionally, LA-RCS interprets natural language commands by the user and converts them into commands compatible with the robot interface so that the robot can execute tasks and meet user requests properly. During this process, the system autonomously evaluates observation results, provides feedback on the tasks, and executes commands on the basis of real-time environmental monitoring, significantly reducing the need for user intervention in fulfilling requests. We categorized the scenarios that LA-RCS is designed to handle into four distinct types and subsequently conducted a quantitative evaluation of its performance across each scenario. The results showed an average success rate of 90%, demonstrating the system’s capability to fulfill user requests satisfactorily. For more extensive results, readers can visit our project website: https://la-rcs.github.io.