Calorie Counter: RGB-Depth Visual Estimation of Energy Expenditure at Home
This work addresses health and lifestyle monitoring by enabling remote calorie tracking without wearable devices, though it is incremental as it builds on vision-based methods for a specific application.
The paper tackled the problem of estimating calorie expenditure from RGB-D data in home settings, achieving accuracy levels above manual metabolic lookup table (MET) estimates and validating on a new dataset with physical gas exchange measurements.
We present a new framework for vision-based estimation of calorific expenditure from RGB-D data - the first that is validated on physical gas exchange measurements and applied to daily living scenarios. Deriving a person's energy expenditure from sensors is an important tool in tracking physical activity levels for health and lifestyle monitoring. Most existing methods use metabolic lookup tables (METs) for a manual estimate or systems with inertial sensors which ultimately require users to wear devices. In contrast, the proposed pose-invariant and individual-independent vision framework allows for a remote estimation of calorific expenditure. We introduce, and evaluate our approach on, a new dataset called SPHERE-calorie, for which visual estimates can be compared against simultaneously obtained, indirect calorimetry measures based on gas exchange. % based on per breath gas exchange. We conclude from our experiments that the proposed vision pipeline is suitable for home monitoring in a controlled environment, with calorific expenditure estimates above accuracy levels of commonly used manual estimations via METs. With the dataset released, our work establishes a baseline for future research for this little-explored area of computer vision.