Topology Distance: A Topology-Based Approach For Evaluating Generative Adversarial Networks
This work addresses the problem of evaluating GANs for researchers in machine learning, offering a novel metric that is incremental as it builds on existing evaluation methods.
The authors tackled the challenge of automatically evaluating Generative Adversarial Networks (GANs) by proposing Topology Distance (TD), a metric based on comparing geometric and topological features of real and generated data manifolds, and demonstrated its superiority over existing metrics like IS, FID, KID, and GS in experiments across various datasets.
Automatic evaluation of the goodness of Generative Adversarial Networks (GANs) has been a challenge for the field of machine learning. In this work, we propose a distance complementary to existing measures: Topology Distance (TD), the main idea behind which is to compare the geometric and topological features of the latent manifold of real data with those of generated data. More specifically, we build Vietoris-Rips complex on image features, and define TD based on the differences in persistent-homology groups of the two manifolds. We compare TD with the most commonly used and relevant measures in the field, including Inception Score (IS), Frechet Inception Distance (FID), Kernel Inception Distance (KID) and Geometry Score (GS), in a range of experiments on various datasets. We demonstrate the unique advantage and superiority of our proposed approach over the aforementioned metrics. A combination of our empirical results and the theoretical argument we propose in favour of TD, strongly supports the claim that TD is a powerful candidate metric that researchers can employ when aiming to automatically evaluate the goodness of GANs' learning.