CTRL Your Shift: Clustered Transfer Residual Learning for Many Small Datasets
This addresses the problem of ensuring accurate and differentiated predictions for practitioners in domains like refugee resettlement, where algorithmic decisions impact placement, though it appears incremental as it builds on existing meta-learning and residual learning techniques.
The paper tackles the challenge of making reliable predictions across many distinct data sources with distributional shifts and varying sample sizes, introducing Clustered Transfer Residual Learning (CTRL) which outperforms state-of-the-art benchmarks on 5 large-scale datasets, including a real-world asylum program dataset.
Machine learning (ML) tasks often utilize large-scale data that is drawn from several distinct sources, such as different locations, treatment arms, or groups. In such settings, practitioners often desire predictions that not only exhibit good overall accuracy, but also remain reliable within each source and preserve the differences that matter across sources. For instance, several asylum and refugee resettlement programs now use ML-based employment predictions to guide where newly arriving families are placed within a host country, which requires generating informative and differentiated predictions for many and often small source locations. However, this task is made challenging by several common characteristics of the data in these settings: the presence of numerous distinct data sources, distributional shifts between them, and substantial variation in sample sizes across sources. This paper introduces Clustered Transfer Residual Learning (CTRL), a meta-learning method that combines the strengths of cross-domain residual learning and adaptive pooling/clustering in order to simultaneously improve overall accuracy and preserve source-level heterogeneity. We provide theoretical results that clarify how our objective navigates the trade-off between data quantity and data quality. We evaluate CTRL alongside other state-of-the-art benchmarks on 5 large-scale datasets. This includes a dataset from the national asylum program in Switzerland, where the algorithmic geographic assignment of asylum seekers is currently being piloted. CTRL consistently outperforms the benchmarks across several key metrics and when using a range of different base learners.