Expressive Monotonic Neural Networks
This work addresses the need for interpretable and fair neural networks in domains like finance and medicine, though it is incremental as it builds on existing monotonicity techniques.
The authors tackled the problem of ensuring monotonic dependence of neural network outputs on specific inputs, proposing a weight-constrained architecture with a residual connection that guarantees exact monotonicity and provides robustness. Their method achieves competitive performance compared to state-of-the-art methods across various benchmarks, including social applications and particle decay classification at CERN.
The monotonic dependence of the outputs of a neural network on some of its inputs is a crucial inductive bias in many scenarios where domain knowledge dictates such behavior. This is especially important for interpretability and fairness considerations. In a broader context, scenarios in which monotonicity is important can be found in finance, medicine, physics, and other disciplines. It is thus desirable to build neural network architectures that implement this inductive bias provably. In this work, we propose a weight-constrained architecture with a single residual connection to achieve exact monotonic dependence in any subset of the inputs. The weight constraint scheme directly controls the Lipschitz constant of the neural network and thus provides the additional benefit of robustness. Compared to currently existing techniques used for monotonicity, our method is simpler in implementation and in theory foundations, has negligible computational overhead, is guaranteed to produce monotonic dependence, and is highly expressive. We show how the algorithm is used to train powerful, robust, and interpretable discriminators that achieve competitive performance compared to current state-of-the-art methods across various benchmarks, from social applications to the classification of the decays of subatomic particles produced at the CERN Large Hadron Collider.