Automatic feature selection and weighting in molecular systems using Differentiable Information Imbalance
This addresses feature selection challenges for researchers in molecular systems and related fields, offering an incremental improvement through automated optimization.
The paper tackles the problem of feature selection and weighting in molecular systems by introducing Differentiable Information Imbalance (DII), an automated method that ranks information content and optimizes feature subsets to preserve relationships in a ground truth space, demonstrating its usefulness on benchmark molecular problems like identifying collective variables and selecting features for machine-learning force fields.
Feature selection is essential in the analysis of molecular systems and many other fields, but several uncertainties remain: What is the optimal number of features for a simplified, interpretable model that retains essential information? How should features with different units be aligned, and how should their relative importance be weighted? Here, we introduce the Differentiable Information Imbalance (DII), an automated method to rank information content between sets of features. Using distances in a ground truth feature space, DII identifies a low-dimensional subset of features that best preserves these relationships. Each feature is scaled by a weight, which is optimized by minimizing the DII through gradient descent. This allows simultaneously performing unit alignment and relative importance scaling, while preserving interpretability. DII can also produce sparse solutions and determine the optimal size of the reduced feature space. We demonstrate the usefulness of this approach on two benchmark molecular problems: (1) identifying collective variables that describe conformations of a biomolecule, and (2) selecting features for training a machine-learning force field. These results show the potential of DII in addressing feature selection challenges and optimizing dimensionality in various applications. The method is available in the Python library DADApy.