Dynamic Algorithms for Online Multiple Testing
This addresses the need for more powerful and flexible statistical methods in fields like genomics and data science, where incremental algorithmic improvements enable dynamic adjustment of testing levels.
The paper tackles the problem of online multiple testing by developing new algorithms that control false discovery exceedance (FDX) and achieve significantly higher power than previous methods, with orders of magnitude improvement in synthetic experiments. It also introduces SupLORD, the first non-trivial algorithm to control false discovery rate (FDR) at stopping times in the online setting.
We derive new algorithms for online multiple testing that provably control false discovery exceedance (FDX) while achieving orders of magnitude more power than previous methods. This statistical advance is enabled by the development of new algorithmic ideas: earlier algorithms are more "static" while our new ones allow for the dynamical adjustment of testing levels based on the amount of wealth the algorithm has accumulated. We demonstrate that our algorithms achieve higher power in a variety of synthetic experiments. We also prove that SupLORD can provide error control for both FDR and FDX, and controls FDR at stopping times. Stopping times are particularly important as they permit the experimenter to end the experiment arbitrarily early while maintaining desired control of the FDR. SupLORD is the first non-trivial algorithm, to our knowledge, that can control FDR at stopping times in the online setting.