Introducing Graph Smoothness Loss for Training Deep Learning Architectures
This work presents an incremental improvement for deep learning practitioners by offering a new loss function that enhances robustness without sacrificing classification accuracy.
The authors tackled the problem of training deep learning architectures for classification by introducing a novel loss function based on graph smoothness, which minimizes label signal smoothness on similarity graphs or maximizes distances between network outputs of different classes. They showed that this loss achieves similar classification performance to cross-entropy while offering additional flexibility and improved robustness to input deviations.
We introduce a novel loss function for training deep learning architectures to perform classification. It consists in minimizing the smoothness of label signals on similarity graphs built at the output of the architecture. Equivalently, it can be seen as maximizing the distances between the network function images of training inputs from distinct classes. As such, only distances between pairs of examples in distinct classes are taken into account in the process, and the training does not prevent inputs from the same class to be mapped to distant locations in the output domain. We show that this loss leads to similar performance in classification as architectures trained using the classical cross-entropy, while offering interesting degrees of freedom and properties. We also demonstrate the interest of the proposed loss to increase robustness of trained architectures to deviations of the inputs.