Quantifying Quality of Class-Conditional Generative Models in Time-Series Domain
This addresses the problem of confidently applying generative models in time-series applications like healthcare and weather forecasting, but it is incremental as it adapts existing image-domain methods.
The paper tackled the lack of consensual quality assessment methods for class-conditional generative models in the time-series domain by introducing the InceptionTime Score (ITS) and Frechet InceptionTime Distance (FITD), and extensive experiments on 80 datasets showed that these metrics combined with TSTR can accurately assess model performance.
Generative models are designed to address the data scarcity problem. Even with the exploding amount of data, due to computational advancements, some applications (e.g., health care, weather forecast, fault detection) still suffer from data insufficiency, especially in the time-series domain. Thus generative models are essential and powerful tools, but they still lack a consensual approach for quality assessment. Such deficiency hinders the confident application of modern implicit generative models on time-series data. Inspired by assessment methods on the image domain, we introduce the InceptionTime Score (ITS) and the Frechet InceptionTime Distance (FITD) to gauge the qualitative performance of class conditional generative models on the time-series domain. We conduct extensive experiments on 80 different datasets to study the discriminative capabilities of proposed metrics alongside two existing evaluation metrics: Train on Synthetic Test on Real (TSTR) and Train on Real Test on Synthetic (TRTS). Extensive evaluation reveals that the proposed assessment method, i.e., ITS and FITD in combination with TSTR, can accurately assess class-conditional generative model performance.