Interestingness First Classifiers
This work addresses the need for novel and interpretable models in knowledge discovery and communication, particularly in domains where moderate accuracy is acceptable, though it is incremental in its approach to feature selection.
The paper tackles the problem of building classifiers that prioritize interestingness over maximum predictive accuracy, introducing the EUREKA framework which selects unusual features using large language models and constructs interpretable classifiers, achieving meaningful accuracy on benchmark datasets like Occupancy Detection and Twin Papers.
Most machine learning models are designed to maximize predictive accuracy. In this work, we explore a different goal: building classifiers that are interesting. An ``interesting classifier'' is one that uses unusual or unexpected features, even if its accuracy is lower than the best possible model. For example, predicting room congestion from CO2 levels achieves near-perfect accuracy but is unsurprising. In contrast, predicting room congestion from humidity is less accurate yet more nuanced and intriguing. We introduce EUREKA, a simple framework that selects features according to their perceived interestingness. Our method leverages large language models to rank features by their interestingness and then builds interpretable classifiers using only the selected interesting features. Across several benchmark datasets, EUREKA consistently identifies features that are non-obvious yet still predictive. For example, in the Occupancy Detection dataset, our method favors humidity over CO2 levels and light intensity, producing classifiers that achieve meaningful accuracy while offering insights. In the Twin Papers dataset, our method discovers the rule that papers with a colon in the title are more likely to be cited in the future. We argue that such models can support new ways of knowledge discovery and communication, especially in settings where moderate accuracy is sufficient but novelty and interpretability are valued.