Black Box FDR
This addresses the need for more powerful statistical analysis in large-scale scientific studies, such as cancer research, though it appears incremental as an extension of existing FDR methods with black box models.
The authors tackled the problem of analyzing multi-experiment studies with many covariates by developing BB-FDR, an empirical-Bayes method that uses black box models to boost power and control false discovery rates, resulting in outperforming state-of-the-art methods in benchmarks and increasing significant outcomes in cancer drug efficacy studies.
Analyzing large-scale, multi-experiment studies requires scientists to test each experimental outcome for statistical significance and then assess the results as a whole. We present Black Box FDR (BB-FDR), an empirical-Bayes method for analyzing multi-experiment studies when many covariates are gathered per experiment. BB-FDR learns a series of black box predictive models to boost power and control the false discovery rate (FDR) at two stages of study analysis. In Stage 1, it uses a deep neural network prior to report which experiments yielded significant outcomes. In Stage 2, a separate black box model of each covariate is used to select features that have significant predictive power across all experiments. In benchmarks, BB-FDR outperforms competing state-of-the-art methods in both stages of analysis. We apply BB-FDR to two real studies on cancer drug efficacy. For both studies, BB-FDR increases the proportion of significant outcomes discovered and selects variables that reveal key genomic drivers of drug sensitivity and resistance in cancer.