Contrastive Image-Metadata Pre-Training for Materials Transmission Electron Microscopy
This work provides a method to leverage unused TEM data for style transfer and denoising, but it is incremental as it applies existing contrastive learning to a specific domain.
The authors introduce a dataset of 7,330 HAADF-STEM images to learn a joint embedding between image metadata and images, enabling style transfer that converts experimental images to different acquisition parameter styles, and they evaluate its use for physical denoising.
The vast majority of transmission electron microscopy (TEM) data never gets published and ends up on a backup drive until deleted to free up space. These left-over datasets are rich in detail and variation, often paired with automatically saved metadata of instrument state and acquisition parameters. In this work, we introduce a dataset of 7,330 high-angle annular dark-field scanning-TEM (HAADF-STEM) images from a single instrument to learn a joint embedding space between image metadata and HAADF image. These embeddings link image style with acquisition parameters, which allows us to train a generative style transfer network that can convert experimental images into the style they would have had if they were recorded with different instrument parameters. We evaluate the performance of the network and explore the usefulness of the technique for physical denoising.