Learning to Partially Defer for Sequences
This addresses the limitation of current L2D methods for sequence prediction tasks, offering a more flexible deferral strategy, though it is incremental as it builds on existing L2D frameworks.
The paper tackles the problem of Learning to Defer (L2D) for sequence outputs by enabling partial deferrals of specific tokens or remaining sequences to experts, rather than deferring entire predictions. The result shows that this granular approach achieves better cost-accuracy tradeoffs in experiments on Traveling Salesman solvers, News summarization, and Weather prediction.
In the Learning to Defer (L2D) framework, a prediction model can either make a prediction or defer it to an expert, as determined by a rejector. Current L2D methods train the rejector to decide whether to reject the {\em entire prediction}, which is not desirable when the model predicts long sequences. We present an L2D setting for sequence outputs where the system can defer \textit{specific outputs} of the whole model prediction to an expert in an effort to interleave the expert and machine throughout the prediction. We propose two types of model-based post-hoc rejectors for pre-trained predictors: a token-level rejector, which defers specific token predictions to experts with next token prediction capabilities, and a one-time rejector for experts without such abilities, which defers the remaining sequence from a specific point onward. In the experiments, we also empirically demonstrate that such granular deferrals achieve better cost-accuracy tradeoffs than whole deferrals on Traveling salesman solvers, News summarization, and Weather prediction.