Feature Selection based on the Local Lift Dependence Scale
This work offers an incremental improvement in feature selection for data analysis, potentially benefiting researchers in fields like education and machine learning by enabling more detailed dependence analysis.
The paper tackles feature selection by extending the search space from Boolean lattices of feature sets to collections of Boolean lattices of ordered pairs, allowing selection of both features and their influential values using a local formulation of mutual information called the Local Lift Dependence Scale. The method is applied to a student performance dataset and two UCI datasets, but no concrete performance numbers are provided.
This paper uses a classical approach to feature selection: minimization of a cost function applied on estimated joint distributions. However, the search space in which such minimization is performed is extended. In the original formulation, the search space is the Boolean lattice of features sets (BLFS), while, in the present formulation, it is a collection of Boolean lattices of ordered pairs (features, associated value) (CBLOP), indexed by the elements of the BLFS. In this approach, we may not only select the features that are most related to a variable Y, but also select the values of the features that most influence the variable or that are most prone to have a specific value of Y. A local formulation of Shanon's mutual information is applied on a CBLOP to select features, namely, the Local Lift Dependence Scale, an scale for measuring variable dependence in multiple resolutions. The main contribution of this paper is to define and apply this local measure, which permits to analyse local properties of joint distributions that are neglected by the classical Shanon's global measure. The proposed approach is applied to a dataset consisting of student performances on a university entrance exam, as well as on undergraduate courses. The approach is also applied to two datasets of the UCI Machine Learning Repository.