Variable Selection in Maximum Mean Discrepancy for Interpretable Distribution Comparison
This work addresses the need for a general framework for interpretable variable selection in distribution comparison, which is incremental as it builds on existing kernel methods but introduces new theoretical and practical contributions.
The paper tackles the problem of identifying which variables cause differences between two datasets, introducing a mathematical definition for the discriminating set and proposing two sparse methods that optimize kernel test power with data-driven regularization, showing improved performance in synthetic experiments and applications to water-pipe and traffic networks.
We study two-sample variable selection: identifying variables that discriminate between the distributions of two sets of data vectors. Such variables help scientists understand the mechanisms behind dataset discrepancies. Although domain-specific methods exist (e.g., in medical imaging, genetics, and computational social science), a general framework remains underdeveloped. We make two separate contributions. (i) We introduce a mathematical notion of the discriminating set of variables: the largest subset containing no variables whose marginals are identical across the two distributions and independent of the remaining variables. We prove this set is uniquely defined and establish further properties, making it a suitable ground truth for theory and evaluation. (ii) We propose two methods for two-sample variable selection that assign weights to variables and optimise them to maximise the power of a kernel two-sample test while enforcing sparsity to downweight redundant variables. To select the regularisation parameter - unknown in practice, as it controls the number of selected variables - we develop two data-driven procedures to balance recall and precision. Synthetic experiments show improved performance over baselines, and we illustrate the approach on two applications using datasets from water-pipe and traffic networks.