Eyeris: Machine Learning Powered Glasses for the Visually Impaired

Abstract

For the over 200 million individuals around the world with visual impairments, navigation remains a persistent challenge due to restricted sensory inputs, dependence on others, and safety risks. Existing technology such as white canes, or GPS-based navigation apps struggle with accurate scene comprehension, contextual awareness, and adaptability to dynamic settings. Recent advancements in embedded GPU acceleration, high-performance vision language models (VLMs), and large language models (LLMs) enable a huge lead in assistive device technologies. This project introduces Eyeris, a wearable assistive glasses system powered by the multimodal vision model Llama3.2-Vision, designed to empower visually impaired individuals through real-time environmental understanding and intuitive audio feedback. Eyeris integrates a 360-degree camera for peripheral vision coverage and Llama3.2-Vision quantized to INT8 precision for low latency and offline personal assistant capabilities. The system processes visual data through a hybrid Vision Transformer (ViT) and transformer-based text encoder, generating contextualizes descriptions of obstacles, signs, and hazards. A voice-activated interface allows natural queries with responses synthesized to speech via the gTTS text-to-speech engine. By merging real-time 360-degree perception with LLM-driven reasoning, Eyeris advances independence, safety, and confidence for visually impaired users, marking a shift in assistive technology. This work highlights the potential of machine learning to reduce accessibility gaps.