GraphMind: Context-Aware Multi-Agent Systems With Graph Attention Autoencoder and Large Language Model Integration

Abstract

In recent years, integrating large language models (LLMs) with multi-agent systems has shown promise in improving the quality and coherence of automated conversations. However, managing the context effectively across multiple agents remains a significant challenge. This project introduces GraphMind, which explores the application of Graph Attention Autoencoder (GAE) to enhance context management and control within multi-agent conversational systems. By leveraging the inherent ability of GAE to focus on relevant parts of a graph-structured input dynamically, our approach aims to track and update the conversation context across agents efficiently. The proposed system integrates LLMs to handle natural language understanding and generation tasks while utilizing GAE to maintain a coherent shared context among agents. Preliminary results demonstrate that this hybrid model improves conversation flow, reduces context-switching errors, and enhances the overall user experience. This work contributes to the field of multi-agent communication by providing a scalable and efficient framework for context management in complex conversational environments. To extend this work into reinforcement learning (RL) in order to increasing the performance, we also focus on utilizing RL to optimize agent behaviors or improve task-specific efficiency dynamically.