PALATE: Peculiar Application of the Law of Total Expectation to Enhance the Evaluation of Deep Generative Models
This work provides a computationally efficient evaluation method for deep generative models, which is important for researchers and practitioners in machine learning, though it appears incremental as it builds on existing metrics like MMD and FLD.
The paper tackles the challenge of comprehensively evaluating deep generative models by proposing PALATE, an enhancement that addresses limitations of existing metrics like Feature Likelihood Divergence, resulting in a framework that matches or surpasses state-of-the-art solutions with superior computational efficiency and scalability to large-scale datasets.
Deep generative models (DGMs) have caused a paradigm shift in the field of machine learning, yielding noteworthy advancements in domains such as image synthesis, natural language processing, and other related areas. However, a comprehensive evaluation of these models that accounts for the trichotomy between fidelity, diversity, and novelty in generated samples remains a formidable challenge. A recently introduced solution that has emerged as a promising approach in this regard is the Feature Likelihood Divergence (FLD), a method that offers a theoretically motivated practical tool, yet also exhibits some computational challenges. In this paper, we propose PALATE, a novel enhancement to the evaluation of DGMs that addresses limitations of existing metrics. Our approach is based on a peculiar application of the law of total expectation to random variables representing accessible real data. When combined with the MMD baseline metric and DINOv2 feature extractor, PALATE offers a holistic evaluation framework that matches or surpasses state-of-the-art solutions while providing superior computational efficiency and scalability to large-scale datasets. Through a series of experiments, we demonstrate the effectiveness of the PALATE enhancement, contributing a computationally efficient, holistic evaluation approach that advances the field of DGMs assessment, especially in detecting sample memorization and evaluating generalization capabilities.