DeepDRK: Deep Dependency Regularized Knockoff for Feature Selection
This work addresses feature selection problems for researchers and practitioners in machine learning, offering improved performance in scenarios with limited data and complex distributions, but it is incremental as it builds on existing knockoff frameworks.
The paper tackled the challenge of maintaining the 'swap property' in deep Model-X knockoff feature selection, which often reduces selection power, by developing DeepDRK, a distribution-free deep learning method that balances false discovery rate (FDR) and power. The result shows that DeepDRK outperforms existing benchmarks across synthetic, semi-synthetic, and real-world datasets, achieving lower FDR and higher power, especially with small sample sizes and non-Gaussian distributions.
Model-X knockoff has garnered significant attention among various feature selection methods due to its guarantees for controlling the false discovery rate (FDR). Since its introduction in parametric design, knockoff techniques have evolved to handle arbitrary data distributions using deep learning-based generative models. However, we have observed limitations in the current implementations of the deep Model-X knockoff framework. Notably, the "swap property" that knockoffs require often faces challenges at the sample level, resulting in diminished selection power. To address these issues, we develop "Deep Dependency Regularized Knockoff (DeepDRK)," a distribution-free deep learning method that effectively balances FDR and power. In DeepDRK, we introduce a novel formulation of the knockoff model as a learning problem under multi-source adversarial attacks. By employing an innovative perturbation technique, we achieve lower FDR and higher power. Our model outperforms existing benchmarks across synthetic, semi-synthetic, and real-world datasets, particularly when sample sizes are small and data distributions are non-Gaussian.