Demystifying Spectral Feature Learning for Instrumental Variable Regression
This provides theoretical insights for researchers in causal inference, but it is incremental as it analyzes an existing spectral feature learning method.
The paper tackles causal effect estimation with hidden confounders using nonparametric instrumental variable regression, deriving a generalization error bound and showing that performance depends on spectral alignment and eigenvalue decay, with synthetic experiments validating a taxonomy of good, bad, and ugly scenarios.
We address the problem of causal effect estimation in the presence of hidden confounders, using nonparametric instrumental variable (IV) regression. A leading strategy employs spectral features - that is, learned features spanning the top eigensubspaces of the operator linking treatments to instruments. We derive a generalization error bound for a two-stage least squares estimator based on spectral features, and gain insights into the method's performance and failure modes. We show that performance depends on two key factors, leading to a clear taxonomy of outcomes. In a good scenario, the approach is optimal. This occurs with strong spectral alignment, meaning the structural function is well-represented by the top eigenfunctions of the conditional operator, coupled with this operator's slow eigenvalue decay, indicating a strong instrument. Performance degrades in a bad scenario: spectral alignment remains strong, but rapid eigenvalue decay (indicating a weaker instrument) demands significantly more samples for effective feature learning. Finally, in the ugly scenario, weak spectral alignment causes the method to fail, regardless of the eigenvalues' characteristics. Our synthetic experiments empirically validate this taxonomy.