Sensing in Smart Cities: A Multimodal Machine Learning Perspective

Abstract

Highlights: What are the main findings? Presents a detailed framework and review of multimodal machine learning (MML) approaches utilized within smart urban environments. Highlights the effectiveness of MML techniques and current technical limitations in modality fusion, scalability, and real-time implementation across urban domains. What are the implications of the main finding? Provides essential guidance to researchers, academicians, policymakers, and developers on choosing effective MML approaches for key smart city applications. Identifies current challenges and practical solutions to advance the deployment of multimodal machine learning in complex urban environments. Smart cities generate vast multimodal data from IoT devices, surveillance systems, health monitors, and environmental monitoring infrastructure. The seamless integration and interpretation of such multimodal data is essential for intelligent decision-making and adaptive urban services. Multimodal machine learning (MML) provides a unified framework to fuse and analyze diverse sources, surpassing conventional unimodal and rule-based approaches. This review surveys the role of MML in smart city sensing across mobility, public safety, healthcare, and environmental domains, outlining key data modalities, enabling technologies and state-of-the-art fusion architectures. We analyze major methodological and deployment challenges, including data alignment, scalability, modality-specific noise, infrastructure limitations, privacy, and ethics, and identify future directions toward scalable, interpretable, and responsible MML for urban systems. This survey serves as a reference for AI researchers, urban planners, and policymakers seeking to understand, design, and deploy multimodal learning solutions for intelligent urban sensing frameworks.