General Framework of AI Agents

Abstract

AI agents represent one of the most prominent frontiers in artificial intelligence. They include software agents that operate on PCs and mobile devices, as well as hardware agents (robots) that function in the physical world. Currently, research and development on AI agents are making significant progress in foundation models, training methods, system design, and practical applications. However, research on agent frameworks remains limited. This article, as a perspective paper, summarizes existing work and proposes a general framework for AI agent information processing. The key features of this framework are that agents are task-oriented, take text and multimodal data as inputs and outputs, use large language models for reasoning, are constructed through reinforcement learning, and leverage tools and memory. Representative agents and agent frameworks developed to date, as well as several agents recently developed at ByteDance Seed, all conform to this general framework. This article introduces these agents as examples. In addition, the article discusses the relationship between the proposed general agent framework and the information processing mechanisms of the human brain, analyzes the main characteristics of agent technologies, and proposes important directions for future research on AI agents.