Accurate Inference for Adaptive Linear Models
This addresses the challenge of accurate inference in adaptive linear models for researchers and practitioners dealing with sequential data collection, offering a novel solution to mitigate biases.
The paper tackles the problem of distributional biases in estimators from adaptively collected data, developing a general W-decorrelation method to transform bias into variance, and demonstrates empirical benefits in multi-armed bandit and autoregressive time series settings with asymptotically correct confidence intervals.
Estimators computed from adaptively collected data do not behave like their non-adaptive brethren. Rather, the sequential dependence of the collection policy can lead to severe distributional biases that persist even in the infinite data limit. We develop a general method -- $\mathbf{W}$-decorrelation -- for transforming the bias of adaptive linear regression estimators into variance. The method uses only coarse-grained information about the data collection policy and does not need access to propensity scores or exact knowledge of the policy. We bound the finite-sample bias and variance of the $\mathbf{W}$-estimator and develop asymptotically correct confidence intervals based on a novel martingale central limit theorem. We then demonstrate the empirical benefits of the generic $\mathbf{W}$-decorrelation procedure in two different adaptive data settings: the multi-armed bandit and the autoregressive time series.