Knowledge-based Document Classification with Shannon Entropy
This provides a simple and explainable solution for document classification, particularly beneficial in domains where positive samples are scarce, though it is incremental as it builds on existing knowledge-based methods.
The paper tackles the problem of noisy detections in knowledge-based document classification by proposing a model that uses Shannon Entropy to favor uniform and diverse keyword matches, resulting in improved recall at a fixed false positive rate and greater robustness against data distribution changes, especially with limited positive training samples.
Document classification is the detection specific content of interest in text documents. In contrast to the data-driven machine learning classifiers, knowledge-based classifiers can be constructed based on domain specific knowledge, which usually takes the form of a collection of subject related keywords. While typical knowledge-based classifiers compute a prediction score based on the keyword abundance, it generally suffers from noisy detections due to the lack of guiding principle in gauging the keyword matches. In this paper, we propose a novel knowledge-based model equipped with Shannon Entropy, which measures the richness of information and favors uniform and diverse keyword matches. Without invoking any positive sample, such method provides a simple and explainable solution for document classification. We show that the Shannon Entropy significantly improves the recall at fixed level of false positive rate. Also, we show that the model is more robust against change of data distribution at inference while compared with traditional machine learning, particularly when the positive training samples are very limited.