Improving Learning-to-Defer Algorithms Through Fine-Tuning
This addresses the problem of optimizing task allocation between humans and AI for practitioners, but it is incremental as it builds on existing learning-to-defer methods.
The paper tackled improving learning-to-defer algorithms for human-AI collaboration by fine-tuning them to specific individuals, finding that fine-tuning captures simple human skill patterns but struggles with nuance.
The ubiquity of AI leads to situations where humans and AI work together, creating the need for learning-to-defer algorithms that determine how to partition tasks between AI and humans. We work to improve learning-to-defer algorithms when paired with specific individuals by incorporating two fine-tuning algorithms and testing their efficacy using both synthetic and image datasets. We find that fine-tuning can pick up on simple human skill patterns, but struggles with nuance, and we suggest future work that uses robust semi-supervised to improve learning.