Rethinking Benchmark and Contamination for Language Models with Rephrased Samples
This addresses the trustworthiness of public benchmarks for the AI research community, highlighting a critical issue with incremental improvements in decontamination methods.
The paper tackles the problem of benchmark contamination in language models, showing that existing decontamination methods are insufficient as simple variations like paraphrasing can bypass them, leading to overfitting and inflated performance (e.g., a 13B model matching GPT-4). It proposes a stronger LLM-based decontamination method, revealing significant overlaps (e.g., 8-18% in HumanEval) in datasets like RedPajama-Data-1T and StarCoder-Data.
Large language models are increasingly trained on all the data ever produced by humans. Many have raised concerns about the trustworthiness of public benchmarks due to potential contamination in pre-training or fine-tuning datasets. While most data decontamination efforts apply string matching (e.g., n-gram overlap) to remove benchmark data, we show that these methods are insufficient, and simple variations of test data (e.g., paraphrasing, translation) can easily bypass these decontamination measures. Furthermore, we demonstrate that if such variation of test data is not eliminated, a 13B model can easily overfit a test benchmark and achieve drastically high performance, on par with GPT-4. We validate such observations in widely used benchmarks such as MMLU, GSK8k, and HumanEval. To address this growing risk, we propose a stronger LLM-based decontamination method and apply it to widely used pre-training and fine-tuning datasets, revealing significant previously unknown test overlap. For example, in pre-training sets such as RedPajama-Data-1T and StarCoder-Data, we identified that 8-18\% of the HumanEval benchmark overlaps. Interestingly, we also find such contamination in synthetic dataset generated by GPT-3.5/4, suggesting a potential risk of unintentional contamination. We urge the community to adopt stronger decontamination approaches when using public benchmarks. Moreover, we call for the community to actively develop fresh one-time exams to evaluate models accurately. Our decontamination tool is publicly available at https://github.com/lm-sys/llm-decontaminator.