AI-native robotic vision systems enabled by in-sensor computing

Abstract

Artificial intelligence (AI) is accelerating the evolution of robotics from task-specific automation to general-purpose autonomy, enabling robots to perform high-level tasks in unstructured and dynamic environments. One of the key enablers in this evolution is the integration of AI with robotic vision systems, which provide accurate perception and contextual interpretation of complex surroundings. An important challenge for this goal is to ensure computational efficiency while robust inference is achieved. One potential solution to tackle this challenge is the acquisition of visual data in formats inherently optimized for AI computing—a concept referred to as AI-native robotic vision. In this review, we highlight recent developments in robotic vision systems with in-sensor computing capabilities. We first discuss the functional features of synapses, neurons, and retinal hierarchies in biological vision systems, and introduce in-sensor computing techniques enabled by device-level emulation of such features. We then present representative studies for each class of in-sensor computing techniques, highlighting their operation principles and robotic vision applications. Finally, we discuss future research directions for advancing in-sensor computing techniques toward the practical implementation of AI-native robotic vision.