Compact Rule-Based Classifier Learning via Gradient Descent
This work addresses the problem of creating interpretable and scalable rule-based classifiers for applications requiring transparency, offering a novel gradient-based approach that improves upon traditional methods.
The paper tackles the challenge of optimizing and scaling rule-based models for high-stakes decision-making by introducing the Fuzzy Rule-based Reasoner (FRR), which uses gradient descent to learn compact, interpretable rules, achieving competitive accuracy with significantly smaller rule bases, such as 96% of the accuracy of state-of-the-art additive models using only 3% of their rule base size.
Rule-based models are essential for high-stakes decision-making due to their transparency and interpretability, but their discrete nature creates challenges for optimization and scalability. In this work, we present the Fuzzy Rule-based Reasoner (FRR), a novel gradient-based rule learning system that supports strict user constraints over rule-based complexity while achieving competitive performance. To maximize interpretability, the FRR uses semantically meaningful fuzzy logic partitions, unattainable with existing neuro-fuzzy approaches, and sufficient (single-rule) decision-making, which avoids the combinatorial complexity of additive rule ensembles. Through extensive evaluation across 40 datasets, FRR demonstrates: (1) superior performance to traditional rule-based methods (e.g., $5\%$ average accuracy over RIPPER); (2) comparable accuracy to tree-based models (e.g., CART) using rule bases $90\%$ more compact; and (3) achieves $96\%$ of the accuracy of state-of-the-art additive rule-based models while using only sufficient rules and requiring only $3\%$ of their rule base size.