MoEcho: Exploiting Side-Channel Attacks to Compromise User Privacy in Mixture-of-Experts LLMs

Abstract

The transformer architecture has become a cornerstone of modern AI, fueling remarkable progress across applications in natural language processing, computer vision, and multi-modal learning. As these models continue to scale explosively for performance, implementation efficiency remains a critical challenge. Mixture-of-Experts (MoE) architectures, selectively activating specialized subnetworks (experts), offer a unique balance between model accuracy and computational cost. However, the adaptive routing in MoE architectures-where input tokens are dynamically directed to specialized experts based on their semantic meaning-inadvertently opens up a new attack surface for privacy breaches. These input-dependent activation patterns leave distinctive temporal and spatial traces in hardware execution, which adversaries could exploit to deduce sensitive user data. In this work, we propose MoEcho (MoE-Echo), discovering a side-channel analysis-based attack surface that compromises user privacy on MoE-based systems. Specifically, in MoEcho, we introduce four novel architectural side-channels on different computing platforms, including Cache Occupancy Channels and Pageout+Reload on CPUs, and Performance Counter and TLB Evict+Reload on GPUs, respectively. Exploiting these vulnerabilities, we propose four attacks that effectively breach user privacy in large-language models (LLMs) and vision-language models (VLMs) based on MoE architectures: Prompt Inference Attack, Response Reconstruction Attack, Visual Inference Attack, and Visual Reconstruction Attack. We evaluate MoEcho on four open-source MoE-based models at different scales, with a specific focus on the DeepSeek architecture. Our end-to-end experiments on both CPU- and GPU-deployed MoE models demonstrate a 99.8% success rate in inferring the patient's private inputs in healthcare records and 92.8% in reconstructing LLM responses. MoEcho is the first run-time architecture-level security analysis of the popular MoE structure common in modern transformers, highlighting a serious security and privacy threat and calling for effective and timely safeguards when harnessing MoE-based models for developing efficient large-scale AI services.