Humans as Light Bulbs: 3D Human Reconstruction from Thermal Reflection
This enables human pose reconstruction in scenarios where direct visual observation is impossible, such as through occlusions or around corners, though it is incremental in leveraging thermal properties for computer vision.
The paper tackles the problem of locating and reconstructing a person's 3D pose using thermal reflections from their body heat, even when they are not visible to a normal camera, by proposing an analysis-by-synthesis framework that jointly models objects, people, and reflections, with experiments showing effectiveness in challenging cases like curved mirrors.
The relatively hot temperature of the human body causes people to turn into long-wave infrared light sources. Since this emitted light has a larger wavelength than visible light, many surfaces in typical scenes act as infrared mirrors with strong specular reflections. We exploit the thermal reflections of a person onto objects in order to locate their position and reconstruct their pose, even if they are not visible to a normal camera. We propose an analysis-by-synthesis framework that jointly models the objects, people, and their thermal reflections, which allows us to combine generative models with differentiable rendering of reflections. Quantitative and qualitative experiments show our approach works in highly challenging cases, such as with curved mirrors or when the person is completely unseen by a normal camera.