Cross-calibration of Time-of-flight and Colour Cameras
This addresses the need for coherent scene representation in automatic scene-interpretation problems, but appears incremental as it builds on existing calibration methods.
The paper tackles the problem of merging depth and color information from time-of-flight and color cameras by developing a geometric framework for multi-view and multi-modal calibration, demonstrating it on a network of three time-of-flight and six color cameras.
Time-of-flight cameras provide depth information, which is complementary to the photometric appearance of the scene in ordinary images. It is desirable to merge the depth and colour information, in order to obtain a coherent scene representation. However, the individual cameras will have different viewpoints, resolutions and fields of view, which means that they must be mutually calibrated. This paper presents a geometric framework for this multi-view and multi-modal calibration problem. It is shown that three-dimensional projective transformations can be used to align depth and parallax-based representations of the scene, with or without Euclidean reconstruction. A new evaluation procedure is also developed; this allows the reprojection error to be decomposed into calibration and sensor-dependent components. The complete approach is demonstrated on a network of three time-of-flight and six colour cameras. The applications of such a system, to a range of automatic scene-interpretation problems, are discussed.