Proper-Composite Loss Functions in Arbitrary Dimensions
This work addresses theoretical foundations for loss function design in ML, offering incremental extensions to existing frameworks.
The paper tackles the problem of analyzing loss functions in machine learning by extending proper-composite representations from finite-dimensional conditional risk to infinite-dimensional total risk minimization, providing a simple characterization of the canonical link.
The study of a machine learning problem is in many ways is difficult to separate from the study of the loss function being used. One avenue of inquiry has been to look at these loss functions in terms of their properties as scoring rules via the proper-composite representation, in which predictions are mapped to probability distributions which are then scored via a scoring rule. However, recent research so far has primarily been concerned with analysing the (typically) finite-dimensional conditional risk problem on the output space, leaving aside the larger total risk minimisation. We generalise a number of these results to an infinite dimensional setting and in doing so we are able to exploit the familial resemblance of density and conditional density estimation to provide a simple characterisation of the canonical link.