Eye2Recall: Exploring Mixed-Initiative Reminiscence Activities via Gaze-Driven LLM Prompts for Older Adults: Eye2Recall: Fusing Gaze and LLMs for Mixed-Initiative Reminiscence with Older Adults

Abstract

Photo-based reminiscence can support well-being in older adults, yet most systems remain text-driven and offer little real-time adaptivity. We first conduct expert interviews to derive design considerations for accessibility, cultural fit, and safe emotional engagement. We then implemented Eye2Recall, an intelligent conversational interface that converts users gazes on old photos into mixed-initiative prompts for a large language model (LLM). We evaluated it in a pilot study with 12 older adults. Participants reported low-effort, smooth interactions, and perceived the agent s questions as aligned with what they were looking at. Immediately after use, self-reported positive mood increased and negative mood decreased. Interviews further indicated that gaze-driven prompts helped retrieve concrete details and supported reflective storytelling. Our contribution is a concrete mechanism for gaze-to-prompt adaptivity that operationalizes mixed-initiative dialogue for older adults reminiscence experience.