MONET -- Virtual Cell Painting of Brightfield Images and Time Lapses Using Reference Consistent Diffusion
This provides a complementary tool for biological research by enabling virtual cell painting to study cell dynamics without chemical fixation, though it is incremental as it builds on existing diffusion models.
The paper tackles the labor-intensive and chemically fixed nature of cell painting by training a diffusion model (MONET) to predict cell paint channels from brightfield images, showing improved quality with scale and enabling time-lapse video generation and in-context learning for out-of-distribution cases.
Cell painting is a popular technique for creating human-interpretable, high-contrast images of cell morphology. There are two major issues with cell paint: (1) it is labor-intensive and (2) it requires chemical fixation, making the study of cell dynamics impossible. We train a diffusion model (Morphological Observation Neural Enhancement Tool, or MONET) on a large dataset to predict cell paint channels from brightfield images. We show that model quality improves with scale. The model uses a consistency architecture to generate time-lapse videos, despite the impossibility of obtaining cell paint video training data. In addition, we show that this architecture enables a form of in-context learning, allowing the model to partially transfer to out-of-distribution cell lines and imaging protocols. Virtual cell painting is not intended to replace physical cell painting completely, but to act as a complementary tool enabling novel workflows in biological research.