Marginal Singularity, and the Benefits of Labels in Covariate-Shift
This work addresses covariate-shift in machine learning, providing theoretical insights for practitioners on when to collect target labels, though it appears incremental in refining existing minimax analyses.
The paper tackles the problem of quantifying the benefits of labeled data under covariate-shift, showing that target labels' effectiveness is controlled by a transfer-exponent γ, which reveals regimes from minimal to dramatic improvements in classification. It also extends a semi-supervised procedure to adapt to unknown γ, achieving minimax transfer rates while using labels only when beneficial.
We present new minimax results that concisely capture the relative benefits of source and target labeled data, under covariate-shift. Namely, we show that the benefits of target labels are controlled by a transfer-exponent $γ$ that encodes how singular Q is locally w.r.t. P, and interestingly allows situations where transfer did not seem possible under previous insights. In fact, our new minimax analysis - in terms of $γ$ - reveals a continuum of regimes ranging from situations where target labels have little benefit, to regimes where target labels dramatically improve classification. We then show that a recently proposed semi-supervised procedure can be extended to adapt to unknown $γ$, and therefore requests labels only when beneficial, while achieving minimax transfer rates.