On the relation between Loss Functions and T-Norms
This work addresses a foundational issue in machine learning by providing a new theoretical framework for loss functions, potentially benefiting all supervised learning tasks, though it appears incremental as it builds on existing t-norm concepts.
The paper tackles the problem of interpreting loss functions in deep learning by establishing a direct relation between loss functions and t-norms, deriving a novel class of loss functions that could achieve faster convergence rates than cross-entropy loss.
Deep learning has been shown to achieve impressive results in several domains like computer vision and natural language processing. A key element of this success has been the development of new loss functions, like the popular cross-entropy loss, which has been shown to provide faster convergence and to reduce the vanishing gradient problem in very deep structures. While the cross-entropy loss is usually justified from a probabilistic perspective, this paper shows an alternative and more direct interpretation of this loss in terms of t-norms and their associated generator functions, and derives a general relation between loss functions and t-norms. In particular, the presented work shows intriguing results leading to the development of a novel class of loss functions. These losses can be exploited in any supervised learning task and which could lead to faster convergence rates that the commonly employed cross-entropy loss.