Reliable Evaluation of Neural Network for Multiclass Classification of Real-world Data
This work addresses the problem of unreliable evaluation metrics for skewed datasets in machine learning, though it is incremental as it applies existing methods to new data.
The paper systematically evaluates a Multilayer Perceptron for multiclass classification on seven real-world datasets using twelve parameters beyond accuracy, revealing insights into their reliability, especially for skewed data.
This paper presents a systematic evaluation of Neural Network (NN) for classification of real-world data. In the field of machine learning, it is often seen that a single parameter that is 'predictive accuracy' is being used for evaluating the performance of a classifier model. However, this parameter might not be considered reliable given a dataset with very high level of skewness. To demonstrate such behavior, seven different types of datasets have been used to evaluate a Multilayer Perceptron (MLP) using twelve(12) different parameters which include micro- and macro-level estimation. In the present study, the most common problem of prediction called 'multiclass' classification has been considered. The results that are obtained for different parameters for each of the dataset could demonstrate interesting findings to support the usability of these set of performance evaluation parameters.