Multi-Task Image-Based Dietary Assessment for Food Recognition and Portion Size Estimation
This work addresses the practical need for integrated dietary assessment tools in nutrition studies, though it represents an incremental improvement over existing single-task approaches.
The authors tackled the problem of image-based dietary assessment by developing a multi-task framework that simultaneously performs food classification and portion size estimation, achieving improved classification accuracy and reduced mean absolute error for portion estimation compared to baseline methods.
Deep learning based methods have achieved impressive results in many applications for image-based diet assessment such as food classification and food portion size estimation. However, existing methods only focus on one task at a time, making it difficult to apply in real life when multiple tasks need to be processed together. In this work, we propose an end-to-end multi-task framework that can achieve both food classification and food portion size estimation. We introduce a food image dataset collected from a nutrition study where the groundtruth food portion is provided by registered dietitians. The multi-task learning uses L2-norm based soft parameter sharing to train the classification and regression tasks simultaneously. We also propose the use of cross-domain feature adaptation together with normalization to further improve the performance of food portion size estimation. Our results outperforms the baseline methods for both classification accuracy and mean absolute error for portion estimation, which shows great potential for advancing the field of image-based dietary assessment.