Implementing the ICE Estimator in Multilayer Perceptron Classifiers
This provides a drop-in replacement for cross-entropy optimization in MLP classifiers to address overfitting, but it is incremental as it applies an existing estimator to a specific framework.
The paper tackles overfitting in multilayer perceptron classifiers by implementing the ICE estimator in Apache Spark's MultilayerPerceptronClassifier, showing it outperforms the stock model using unadjusted MLE loss in cross-validation with identical runtime and similar fitting performance.
This paper describes the techniques used to implement the ICE estimator for a multilayer perceptron model, and reviews the performance of the resulting models. The ICE estimator is implemented in the Apache Spark MultilayerPerceptronClassifier, and shown in cross-validation to outperform the stock MultilayerPerceptronClassifier that uses unadjusted MLE (cross-entropy) loss. The resulting models have identical runtime performance, and similar fitting performance to the stock MLP implementations. Additionally, this approach requires no hyper-parameters, and is therefore viable as a drop-in replacement for cross-entropy optimizing multilayer perceptron classifiers wherever overfitting may be a concern.