Coordinated Physics-Informed Multi-Agent Reinforcement Learning for Risk-Aware Supply Chain Optimization

Abstract

Multi-agent reinforcement learning for supply chain optimization faces significant challenges in maintaining physical consistency, quantifying operational risks, and coordinating distributed decision-making. Existing approaches frequently violate fundamental conservation principles, rely on expected value optimization that ignores tail risks, and lack effective mechanisms for ensuring global coherence across autonomous agents. We present PIMA-DRL, a unified framework that combines physics-informed neural networks with distributional reinforcement learning to simultaneously address these limitations. The framework enforces conservation laws through differentiable physics constraints embedded directly into the learning process, while maintaining complete probability distributions over returns to enable sophisticated risk assessment through Conditional Value-at-Risk optimization. A novel coordination mechanism based on Lagrangian duality ensures global consistency while preserving agent autonomy through decentralized multiplier updates. We establish theoretical convergence guarantees under bounded physics constraint violations and prove enhanced sample efficiency compared to physics-agnostic methods. Local agent states encompass inventory levels, flow dynamics, demand projections, and capacity limitations, with evolution governed by conservation-based differential equations. Comprehensive experiments across multi-echelon inventory networks, hub-spoke distribution systems, and perishable goods supply chains demonstrate substantial improvements over conventional baselines. Results show conservation error reductions of 87.3%, constraint violation decreases of 74.6%, and tail risk improvements of 42.1% while achieving 2.4x faster convergence. The physics-informed structure enhances model interpretability and provides principled uncertainty quantification critical for robust supply chain management under operational variability.