Voice-Assisted Food Recall Using Voice Assistants

Abstract

In this paper, we design a voice-assisted food recall tool that can be implemented on voice assistants of smart speakers and smartphones, enabling frequent, quick, and real-time self-administered food recall. We envision that voice-assisted food recall can improve the accuracy and usability of the web-based Automated Self-administered 24-h Assessment (ASA-24). ASA-24 was developed by the National Cancer Institute in 2010 and has been widely used in clinical and research settings, but has low compliance and completion rates for at-home users. Specifically, we designed a prototype using nine ASA-24 general questions, two free-recall questions, five ASA-24 detailed questions, and three clarifying strategies. The integration of the ASA-24 questions ensures that the output of the prototype will align with the output of the ASA-24, so as to connect to the ASA-24 for nutrition profile analysis. We recruited twenty young adults and twenty older adults to evaluate this prototype, with each using it to recall three meals. We evaluated participants’ performance per different types of questions and strategies, and analyzed the strength and weaknesses of the voice-assisted food recall. The mean success rate and session time of a single meal was (96.4%, 141.4 s) for young adults and (88.6%, 165.4 s) for older adults. The voice-assisted recall session time is significantly shorter than the ASA-24 single-meal session time, and the relevance of voice responses are determined to be high. We conducted questionnaires and interviews to obtain participants’ feedback on the feasibility and acceptability of the prototype. 65% of young and 60% of older participants prefer voice-assisted food recall over web-based food recall, showing promising feasibility and acceptance of our initial voice-assisted food recall prototype. Future works include validation of the food recall content, development of food-customized speech recognition and natural language understanding techniques to enhance accuracy, and integration of human-like assistance to improve usability.