Improving the Evaluation of Generative Models with Fuzzy Logic
This work addresses the need for objective and interpretable metrics in AI evaluation, specifically for image generation, offering a domain-specific improvement over existing methods.
The paper tackles the problem of evaluating image generation tasks by proposing a novel metric called Fuzzy Topology Impact (FTI), which uses topology and fuzzy logic to assess quality and diversity, showing better and more stable performance in experiments on sensitivity to noise, mode dropping, and mode inventing.
Objective and interpretable metrics to evaluate current artificial intelligent systems are of great importance, not only to analyze the current state of such systems but also to objectively measure progress in the future. In this work, we focus on the evaluation of image generation tasks. We propose a novel approach, called Fuzzy Topology Impact (FTI), that determines both the quality and diversity of an image set using topology representations combined with fuzzy logic. When compared to current evaluation methods, FTI shows better and more stable performance on multiple experiments evaluating the sensitivity to noise, mode dropping and mode inventing.