Accelerated Aggregated D-Optimal Designs for Estimating Main Effects in Black-Box Models
This work addresses the need for scalable and robust explanation methods in machine learning, particularly for interpreting black-box models, though it appears incremental as it builds on existing experimental design principles.
The paper tackles the problem of estimating main effects in black-box models, which is challenging due to scalability, sensitivity to out-of-distribution sampling, and instability under correlated features, and proposes A2D2E, an estimator based on accelerated aggregated D-optimal designs, achieving improved efficiency and robustness with theoretical guarantees and validation through simulations and a case study.
Recent advances in supervised learning have driven growing interest in explaining black-box models, particularly by estimating the effects of input variables on model predictions. However, existing approaches often face key limitations, including poor scalability, sensitivity to out-of-distribution sampling, and instability under correlated features. To address these issues, we propose A2D2E, an $\textbf{E}$stimator based on $\textbf{A}$ccelerated $\textbf{A}$ggregated $\textbf{D}$-Optimal $\textbf{D}$esigns. Our method leverages principled experimental design to improve efficiency and robustness in main effect estimation. We establish theoretical guarantees, including convergence and variance reduction, and validate A2D2E through extensive simulations. We further provide the potential of the proposed method with a case study on real data and applications in language models. The code to reproduce the results can be found at https://github.com/cchihyu/A2D2E.