Computer-Assisted Creation of Boolean Search Rules for Text Classification in the Legal Domain
This addresses the need for interpretable classifiers in the legal domain, though it is incremental as it builds on existing Boolean rule methods with interactive guidance.
The paper tackles the problem of creating explainable text classifiers by developing an interactive system, CASE, that assists human annotators in building Boolean search rules using word co-occurrence, and evaluates these rules on 4 datasets, showing trade-offs in compactness and simplicity versus performance compared to machine learning models like Random Forest and fastText.
In this paper, we present a method of building strong, explainable classifiers in the form of Boolean search rules. We developed an interactive environment called CASE (Computer Assisted Semantic Exploration) which exploits word co-occurrence to guide human annotators in selection of relevant search terms. The system seamlessly facilitates iterative evaluation and improvement of the classification rules. The process enables the human annotators to leverage the benefits of statistical information while incorporating their expert intuition into the creation of such rules. We evaluate classifiers created with our CASE system on 4 datasets, and compare the results to machine learning methods, including SKOPE rules, Random forest, Support Vector Machine, and fastText classifiers. The results drive the discussion on trade-offs between superior compactness, simplicity, and intuitiveness of the Boolean search rules versus the better performance of state-of-the-art machine learning models for text classification.