Task Specific Adversarial Cost Function
This work addresses the need for adaptable cost functions in generative models to enhance task-specific performance, representing an incremental improvement over standard adversarial training methods.
The paper tackles the problem of designing cost functions for generative models tailored to specific tasks like generation or retrieval, proposing a task-specific adversarial cost function that allows tuning between different Kullback-Leibler divergences. It evaluates this approach on hand-written character datasets for generation, retrieval, and one-shot learning tasks, showing improved performance in these areas.
The cost function used to train a generative model should fit the purpose of the model. If the model is intended for tasks such as generating perceptually correct samples, it is beneficial to maximise the likelihood of a sample drawn from the model, Q, coming from the same distribution as the training data, P. This is equivalent to minimising the Kullback-Leibler (KL) distance, KL[Q||P]. However, if the model is intended for tasks such as retrieval or classification it is beneficial to maximise the likelihood that a sample drawn from the training data is captured by the model, equivalent to minimising KL[P||Q]. The cost function used in adversarial training optimises the Jensen-Shannon entropy which can be seen as an even interpolation between KL[Q||P] and KL[P||Q]. Here, we propose an alternative adversarial cost function which allows easy tuning of the model for either task. Our task specific cost function is evaluated on a dataset of hand-written characters in the following tasks: Generation, retrieval and one-shot learning.