Revealing Microscopic Objects in Fluorescence Live Imaging by Video-to-video Translation Based on A Spatial-temporal Generative Adversarial Network
This work addresses a domain-specific problem in fluorescence live imaging for researchers, offering an incremental improvement by applying an existing GAN framework to a new application area.
The paper tackles the limitation of standard fluorescence microscopy in visualizing multiple subcellular structures due to spectral conflicts from limited fluorescent labels, proposing a video-to-video translation method based on a Spatial-temporal Generative Adversarial Network (STGAN) to reveal spatial-temporal relationships and translate videos between domains, with experimental results confirming its effectiveness in mitigating these conflicts and enabling simultaneous visualization of multiple microscopic objects.
In spite of being a valuable tool to simultaneously visualize multiple types of subcellular structures using spectrally distinct fluorescent labels, a standard fluoresce microscope is only able to identify a few microscopic objects; such a limit is largely imposed by the number of fluorescent labels available to the sample. In order to simultaneously visualize more objects, in this paper, we propose to use video-to-video translation that mimics the development process of microscopic objects. In essence, we use a microscopy video-to-video translation framework namely Spatial-temporal Generative Adversarial Network (STGAN) to reveal the spatial and temporal relationships between the microscopic objects, after which a microscopy video of one object can be translated to another object in a different domain. The experimental results confirm that the proposed STGAN is effective in microscopy video-to-video translation that mitigates the spectral conflicts caused by the limited fluorescent labels, allowing multiple microscopic objects be simultaneously visualized.