Maximum mutual information regularized classification
This work addresses pattern classification problems for researchers and practitioners, offering an incremental improvement by integrating mutual information regularization into existing classifier frameworks.
The paper tackles pattern classification by proposing a novel approach that regularizes classifier learning to maximize mutual information between classification responses and true class labels, aiming to reduce uncertainty in predictions. Experiments on two real-world problems demonstrate significant improvements, though specific numerical gains are not detailed.
In this paper, a novel pattern classification approach is proposed by regularizing the classifier learning to maximize mutual information between the classification response and the true class label. We argue that, with the learned classifier, the uncertainty of the true class label of a data sample should be reduced by knowing its classification response as much as possible. The reduced uncertainty is measured by the mutual information between the classification response and the true class label. To this end, when learning a linear classifier, we propose to maximize the mutual information between classification responses and true class labels of training samples, besides minimizing the classification error and reduc- ing the classifier complexity. An objective function is constructed by modeling mutual information with entropy estimation, and it is optimized by a gradi- ent descend method in an iterative algorithm. Experiments on two real world pattern classification problems show the significant improvements achieved by maximum mutual information regularization.