Self-Explaining Neural Networks for Food Recognition and Dietary Analysis

Abstract

Food pattern recognition plays a crucial role in modern healthcare by enabling automated dietary monitoring and personalised nutritional interventions, particularly for vulnerable populations with complex dietary needs. Current food recognition systems struggle to balance high accuracy with interpretability and computational efficiency when analysing complex meal compositions in real-world settings. We developed a novel self-explaining neural architecture that integrates specialised attention mechanisms with temporal modules within a streamlined framework. Our methodology employs hierarchical feature extraction through successive convolution operations, multi-head attention mechanisms for pattern classification, and bidirectional LSTM networks for temporal analysis. Architecture incorporates self-explaining components utilising attention-based mechanisms and interpretable concept encoders to maintain transparency. We evaluated our model on the FOOD101 dataset using 5-fold cross-validation, ablation studies, and comprehensive computational efficiency assessments. Training employed multi-objective optimisation with adaptive learning rates and specialised loss functions designed for dietary pattern recognition. Experiments demonstrate our model’s superior performance, achieving 94.1% accuracy with only 29.3 ms inference latency and 3.8 GB memory usage, representing a 63.3% parameter reduction compared to baseline transformers. The system maintains detection rates above 84% in complex multi-item recognition scenarios, whilst feature attribution analysis achieved scores of 0.89 for primary components. Cross-validation confirmed consistent performance with accuracy ranging from 92.8% to 93.5% across all folds. This research advances automated dietary analysis by providing an efficient, interpretable solution for food recognition with direct applications in nutritional monitoring and personalised healthcare, particularly benefiting vulnerable populations who require transparent and trustworthy dietary guidance.