Copula Representations and Error Surface Projections for the Exclusive Or Problem
This is an incremental pedagogical work aimed at beginners in machine learning to illustrate fundamental concepts.
The paper tackles the representation and approximation of the exclusive or (xor) function using probabilistic logic and associative copula functions, extending it from Boolean to real values to demonstrate cross-validation concepts, with results illustrated through error surface charts for different activation functions like RELU and tanh.
The exclusive or (xor) function is one of the simplest examples that illustrate why nonlinear feedforward networks are superior to linear regression for machine learning applications. We review the xor representation and approximation problems and discuss their solutions in terms of probabilistic logic and associative copula functions. After briefly reviewing the specification of feedforward networks, we compare the dynamics of learned error surfaces with different activation functions such as RELU and tanh through a set of colorful three-dimensional charts. The copula representations extend xor from Boolean to real values, thereby providing a convenient way to demonstrate the concept of cross-validation on in-sample and out-sample data sets. Our approach is pedagogical and is meant to be a machine learning prolegomenon.