Dimension Reduction by Mutual Information Discriminant Analysis
This work addresses feature extraction for high-dimensional data analysis, offering an incremental improvement by applying mutual information in a new way to a known bottleneck in discriminant analysis.
The paper tackles the problem of feature extraction in high-dimensional data by proposing a novel discriminant analysis algorithm based on mutual information, called MIDA, which uses one-dimensional estimations to overcome challenges in high-dimensional MI. Results on UCI databases show that MIDA provides robust performance, performing better than or comparable to the best existing algorithms across different datasets.
In the past few decades, researchers have proposed many discriminant analysis (DA) algorithms for the study of high-dimensional data in a variety of problems. Most DA algorithms for feature extraction are based on transformations that simultaneously maximize the between-class scatter and minimize the withinclass scatter matrices. This paper presents a novel DA algorithm for feature extraction using mutual information (MI). However, it is not always easy to obtain an accurate estimation for high-dimensional MI. In this paper, we propose an efficient method for feature extraction that is based on one-dimensional MI estimations. We will refer to this algorithm as mutual information discriminant analysis (MIDA). The performance of this proposed method was evaluated using UCI databases. The results indicate that MIDA provides robust performance over different data sets with different characteristics and that MIDA always performs better than, or at least comparable to, the best performing algorithms.