A Survey on Evaluation Metrics for Synthetic Material Micro-Structure Images from Generative Models
This addresses the problem of evaluating synthetic images for materials scientists, but it is incremental as it surveys and critiques existing methods without proposing new solutions.
The paper evaluated existing methods for assessing synthetic micro-structure images, such as Fréchet Inception Distance, on SEM images of graphene-reinforced polyurethane foams, finding them limited due to unique material features and small datasets, and aimed to highlight these shortcomings to encourage metric improvements in materials science.
The evaluation of synthetic micro-structure images is an emerging problem as machine learning and materials science research have evolved together. Typical state of the art methods in evaluating synthetic images from generative models have relied on the Fréchet Inception Distance. However, this and other similar methods, are limited in the materials domain due to both the unique features that characterize physically accurate micro-structures and limited dataset sizes. In this study we evaluate a variety of methods on scanning electron microscope (SEM) images of graphene-reinforced polyurethane foams. The primary objective of this paper is to report our findings with regards to the shortcomings of existing methods so as to encourage the machine learning community to consider enhancements in metrics for assessing quality of synthetic images in the material science domain.