Estimating and abstracting the 3D structure of bones using neural networks on X-ray (2D) images
This provides a fully automated, geometry-free method for forensic bone database searches, though it is incremental as it builds on existing 3D reconstruction techniques.
The paper tackles the problem of estimating 3D bone structure from 2D X-ray images using a deep-learning method, achieving an average RMS distance of 1.08 mm, which is more accurate than eight other approaches, and enables 100% accurate bone identification via embeddings.
In this paper, we present a deep-learning based method for estimating the 3D structure of a bone from a pair of 2D X-ray images. Our triplet loss-trained neural network selects the most closely matching 3D bone shape from a predefined set of shapes. Our predictions have an average root mean square (RMS) distance of 1.08 mm between the predicted and true shapes, making it more accurate than the average error achieved by eight other examined 3D bone reconstruction approaches. The prediction process that we use is fully automated and unlike many competing approaches, it does not rely on any previous knowledge about bone geometry. Additionally, our neural network can determine the identity of a bone based only on its X-ray image. It computes a low-dimensional representation ("embedding") of each 2D X-ray image and henceforth compares different X-ray images based only on their embeddings. An embedding holds enough information to uniquely identify the bone CT belonging to the input X-ray image with a 100% accuracy and can therefore serve as a kind of fingerprint for that bone. Possible applications include faster, image content-based bone database searches for forensic purposes.