Feature Selection via Probabilistic Outputs
This work addresses feature selection for machine learning practitioners, but it is incremental as it builds on existing methods with theoretical analysis and empirical validation.
The paper tackles feature selection by analyzing two scoring criteria based on estimated class probabilities, predicting when each is advantageous and validating these predictions empirically.
This paper investigates two feature-scoring criteria that make use of estimated class probabilities: one method proposed by \citet{shen} and a complementary approach proposed below. We develop a theoretical framework to analyze each criterion and show that both estimate the spread (across all values of a given feature) of the probability that an example belongs to the positive class. Based on our analysis, we predict when each scoring technique will be advantageous over the other and give empirical results validating our predictions.