A Diffusion Model Predicts 3D Shapes from 2D Microscopy Images
This addresses the inverse biomedical problem of 3D shape reconstruction from 2D images, with incremental improvements in data augmentation for imbalanced cell classification.
The paper tackles the problem of predicting 3D cell shapes from 2D microscopy images using a diffusion model called DISPR, and shows that using its predictions for data augmentation improves the macro F1 score in a classification task from 55.2% to 72.2%.
Diffusion models are a special type of generative model, capable of synthesising new data from a learnt distribution. We introduce DISPR, a diffusion-based model for solving the inverse problem of three-dimensional (3D) cell shape prediction from two-dimensional (2D) single cell microscopy images. Using the 2D microscopy image as a prior, DISPR is conditioned to predict realistic 3D shape reconstructions. To showcase the applicability of DISPR as a data augmentation tool in a feature-based single cell classification task, we extract morphological features from the red blood cells grouped into six highly imbalanced classes. Adding features from the DISPR predictions to the three minority classes improved the macro F1 score from $F1_\text{macro} = 55.2 \pm 4.6\%$ to $F1_\text{macro} = 72.2 \pm 4.9\%$. We thus demonstrate that diffusion models can be successfully applied to inverse biomedical problems, and that they learn to reconstruct 3D shapes with realistic morphological features from 2D microscopy images.