On the Predictive Properties of Binary Link Functions
This provides theoretical justification for a long-held but unproven claim in binary classification, which is incremental but clarifies foundational assumptions for researchers and practitioners.
The paper formally studies the similarities and differences between binary link functions like probit and logit, proving that they are perfectly predictively concordant and showing high predictive equivalence for others like cauchit and complementary log log.
This paper provides a theoretical and computational justification of the long held claim that of the similarity of the probit and logit link functions often used in binary classification. Despite this widespread recognition of the strong similarities between these two link functions, very few (if any) researchers have dedicated time to carry out a formal study aimed at establishing and characterizing firmly all the aspects of the similarities and differences. This paper proposes a definition of both structural and predictive equivalence of link functions-based binary regression models, and explores the various ways in which they are either similar or dissimilar. From a predictive analytics perspective, it turns out that not only are probit and logit perfectly predictively concordant, but the other link functions like cauchit and complementary log log enjoy very high percentage of predictive equivalence. Throughout this paper, simulated and real life examples demonstrate all the equivalence results that we prove theoretically.