Predictive Inference in Multi-environment Scenarios
This work addresses predictive inference challenges in multi-environment scenarios, which is incremental but important for applications like neurochemical sensing and species classification.
The paper tackles the problem of constructing valid confidence intervals and sets for prediction across multiple environments, extending jackknife and split-conformal methods to achieve distribution-free coverage and introducing a novel resizing method that reduces prediction set sizes using limited test environment information, as evaluated on neurochemical sensing and species classification datasets.
We address the challenge of constructing valid confidence intervals and sets in problems of prediction across multiple environments. We investigate two types of coverage suitable for these problems, extending the jackknife and split-conformal methods to show how to obtain distribution-free coverage in such non-traditional, potentially hierarchical data-generating scenarios. We demonstrate a novel resizing method to adapt to problem difficulty, which applies both to existing approaches for predictive inference and the methods we develop; this reduces prediction set sizes using limited information from the test environment, a key to the methods' practical performance, which we evaluate through neurochemical sensing and species classification datasets. Our contributions also include extensions for settings with non-real-valued responses, a theory of consistency for predictive inference in these general problems, and insights on the limits of conditional coverage.