On the alpha-loss Landscape in the Logistic Model
This work provides insights into optimization challenges for practitioners using α-loss in emerging learning methods, but it is incremental as it builds on existing loss function analysis.
The paper analyzes the optimization landscape of the tunable α-loss family in logistic models, showing how the landscape evolves with α and interpreting the results in terms of optimization complexity via normalized gradient descent.
We analyze the optimization landscape of a recently introduced tunable class of loss functions called $α$-loss, $α\in (0,\infty]$, in the logistic model. This family encapsulates the exponential loss ($α= 1/2$), the log-loss ($α= 1$), and the 0-1 loss ($α= \infty$) and contains compelling properties that enable the practitioner to discern among a host of operating conditions relevant to emerging learning methods. Specifically, we study the evolution of the optimization landscape of $α$-loss with respect to $α$ using tools drawn from the study of strictly-locally-quasi-convex functions in addition to geometric techniques. We interpret these results in terms of optimization complexity via normalized gradient descent.