In-Context Learning (and Unlearning) of Length Biases
This addresses robustness issues in language models for NLP practitioners by revealing and mitigating statistical biases, though it is incremental as it builds on prior work on lexical and label biases.
The paper investigates how large language models learn length biases during in-context learning, showing that models adopt these biases from exemplars and that in-context learning can counteract biases from fine-tuning without parameter updates.
Large language models have demonstrated strong capabilities to learn in-context, where exemplar input-output pairings are appended to the prompt for demonstration. However, existing work has demonstrated the ability of models to learn lexical and label biases in-context, which negatively impacts both performance and robustness of models. The impact of other statistical data biases remains under-explored, which this work aims to address. We specifically investigate the impact of length biases on in-context learning. We demonstrate that models do learn length biases in the context window for their predictions, and further empirically analyze the factors that modulate the level of bias exhibited by the model. In addition, we show that learning length information in-context can be used to counter the length bias that has been encoded in models (e.g., via fine-tuning). This reveals the power of in-context learning in debiasing model prediction behaviors without the need for costly parameter updates.