An Extension of Fisher's Criterion: Theoretical Results with a Neural Network Realization
This work addresses a specific bottleneck in feature selection for classification tasks, but appears incremental as it extends an existing criterion.
The authors tackled the limitation of Fisher's criterion performing poorly when class means are close by proposing an extension that leverages heteroscedasticity, and demonstrated its viability in a proof-of-concept neural network experiment.
Fisher's criterion is a widely used tool in machine learning for feature selection. For large search spaces, Fisher's criterion can provide a scalable solution to select features. A challenging limitation of Fisher's criterion, however, is that it performs poorly when mean values of class-conditional distributions are close to each other. Motivated by this challenge, we propose an extension of Fisher's criterion to overcome this limitation. The proposed extension utilizes the available heteroscedasticity of class-conditional distributions to distinguish one class from another. Additionally, we describe how our theoretical results can be casted into a neural network framework, and conduct a proof-of-concept experiment to demonstrate the viability of our approach to solve classification problems.