Quantifying Overfitting: Introducing the Overfitting Index
This addresses the challenge of model generalizability for machine learning practitioners by providing an objective tool to gauge overfitting, though it appears incremental as it builds on existing metrics and datasets.
The paper tackles the problem of overfitting in machine learning by introducing the Overfitting Index (OI), a novel metric to quantitatively assess a model's tendency to overfit, with experiments on datasets like Breast Ultrasound Images and MNIST showing variable overfitting behaviors across architectures and the impact of data augmentation.
In the rapidly evolving domain of machine learning, ensuring model generalizability remains a quintessential challenge. Overfitting, where a model exhibits superior performance on training data but falters on unseen data, is a recurrent concern. This paper introduces the Overfitting Index (OI), a novel metric devised to quantitatively assess a model's tendency to overfit. Through extensive experiments on the Breast Ultrasound Images Dataset (BUS) and the MNIST dataset using architectures such as MobileNet, U-Net, ResNet, Darknet, and ViT-32, we illustrate the utility and discernment of the OI. Our results underscore the variable overfitting behaviors across architectures and highlight the mitigative impact of data augmentation, especially on smaller and more specialized datasets. The ViT-32's performance on MNIST further emphasizes the robustness of certain models and the dataset's comprehensive nature. By providing an objective lens to gauge overfitting, the OI offers a promising avenue to advance model optimization and ensure real-world efficacy.