What Happened 3 Seconds Ago? Inferring the Past with Thermal Imaging
This work addresses a challenging task in human motion analysis for computer vision applications, but it is incremental as it applies an existing method to a new data type.
The paper tackles the problem of inferring past human motion by introducing thermal imaging to reduce prediction uncertainty, achieving remarkable performance through a new dataset and model.
Inferring past human motion from RGB images is challenging due to the inherent uncertainty of the prediction problem. Thermal images, on the other hand, encode traces of past human-object interactions left in the environment via thermal radiation measurement. Based on this observation, we collect the first RGB-Thermal dataset for human motion analysis, dubbed Thermal-IM. Then we develop a three-stage neural network model for accurate past human pose estimation. Comprehensive experiments show that thermal cues significantly reduce the ambiguities of this task, and the proposed model achieves remarkable performance. The dataset is available at https://github.com/ZitianTang/Thermal-IM.