goFOODTM: An artificial intelligence system for dietary assessment

Abstract

Accurate estimation of nutritional information may lead to healthier diets and better clinical outcomes. We propose a dietary assessment system based on artificial intelligence (AI), named goFOOD™. The system can estimate the calorie and macronutrient content of a meal, on the sole basis of food images captured by a smartphone. goFOOD™ requires an input of two meal images or a short video. For conventional single-camera smartphones, the images must be captured from two different viewing angles; smartphones equipped with two rear cameras require only a single press of the shutter button. The deep neural networks are used to process the two images and implements food detection, segmentation and recognition, while a 3D reconstruction algorithm estimates the food’s volume. Each meal’s calorie and macronutrient content is calculated from the food category, volume and the nutrient database. goFOOD™ supports 319 fine-grained food categories, and has been validated on two multimedia databases that contain non-standardized and fast food meals. The experimental results demonstrate that goFOOD™ performed better than experienced dietitians on the non-standardized meal database, and was comparable to them on the fast food database. goFOOD™ provides a simple and efficient solution to the end-user for dietary assessment.