Adversarial training for predictive tasks: theoretical analysis and limitations in the deterministic case
This work addresses a specific limitation in adversarial training for deterministic tasks, offering incremental insights for researchers in machine learning applied to geophysical data.
The paper investigates why adversarial training fails to improve results for deterministic sequences in geophysical processing, providing a theoretical explanation and proposing an adversarial content loss that yields better performance on their data.
To train a deep neural network to mimic the outcomes of processing sequences, a version of Conditional Generalized Adversarial Network (CGAN) can be used. It has been observed by others that CGAN can help to improve the results even for deterministic sequences, where only one output is associated with the processing of a given input. Surprisingly, our CGAN-based tests on deterministic geophysical processing sequences did not produce a real improvement compared to the use of an $L_p$ loss; we here propose a first theoretical explanation why. Our analysis goes from the non-deterministic case to the deterministic one. It led us to develop an adversarial way to train a content loss that gave better results on our data.