Two-view 3D Reconstruction for Food Volume Estimation
This addresses the need for automated dietary assessment tools to help individuals manage diet-related chronic diseases, though it is incremental as it builds on existing computer vision methods.
The paper tackles the problem of food portion estimation by proposing a three-stage system that uses two images from mobile devices to reconstruct a dense 3D model and extract volume, achieving an average error of less than 10% on 77 real dishes in 5.5 seconds per dish.
The increasing prevalence of diet-related chronic diseases coupled with the ineffectiveness of traditional diet management methods have resulted in a need for novel tools to accurately and automatically assess meals. Recently, computer vision based systems that use meal images to assess their content have been proposed. Food portion estimation is the most difficult task for individuals assessing their meals and it is also the least studied area. The present paper proposes a three-stage system to calculate portion sizes using two images of a dish acquired by mobile devices. The first stage consists in understanding the configuration of the different views, after which a dense 3D model is built from the two images; finally, this 3D model serves to extract the volume of the different items. The system was extensively tested on 77 real dishes of known volume, and achieved an average error of less than 10% in 5.5 seconds per dish. The proposed pipeline is computationally tractable and requires no user input, making it a viable option for fully automated dietary assessment.