Evaluation of Machine-generated Biomedical Images via A Tally-based Similarity Measure
This work addresses the critical need for robust evaluation in mission-critical biomedical image synthesis, though it is incremental as it applies an existing similarity measure to this domain.
The paper tackled the problem of quantitatively evaluating machine-generated biomedical images where ground truth is unavailable, demonstrating that using the Tversky Index for perceptual similarity leads to more intuitive results compared to traditional deep feature distance methods.
Super-resolution, in-painting, whole-image generation, unpaired style-transfer, and network-constrained image reconstruction each include an aspect of machine-learned image synthesis where the actual ground truth is not known at time of use. It is generally difficult to quantitatively and authoritatively evaluate the quality of synthetic images; however, in mission-critical biomedical scenarios robust evaluation is paramount. In this work, all practical image-to-image comparisons really are relative qualifications, not absolute difference quantifications; and, therefore, meaningful evaluation of generated image quality can be accomplished using the Tversky Index, which is a well-established measure for assessing perceptual similarity. This evaluation procedure is developed and then demonstrated using multiple image data sets, both real and simulated. The main result is that when the subjectivity and intrinsic deficiencies of any feature-encoding choice are put upfront, Tversky's method leads to intuitive results, whereas traditional methods based on summarizing distances in deep feature spaces do not.