An Assessment of Model-On-Model Deception
This addresses the trustworthiness and reliability of language models for users and developers, but it is incremental as it assesses an existing risk without introducing new defenses.
The paper tackled the problem of language models producing deceptive outputs that mislead other models, by creating a dataset of over 10,000 misleading explanations from models like Llama-2 and GPT-3.5 on MMLU questions, and found that all models tested were significantly deceived, with more capable models only slightly better at resisting deception.
The trustworthiness of highly capable language models is put at risk when they are able to produce deceptive outputs. Moreover, when models are vulnerable to deception it undermines reliability. In this paper, we introduce a method to investigate complex, model-on-model deceptive scenarios. We create a dataset of over 10,000 misleading explanations by asking Llama-2 7B, 13B, 70B, and GPT-3.5 to justify the wrong answer for questions in the MMLU. We find that, when models read these explanations, they are all significantly deceived. Worryingly, models of all capabilities are successful at misleading others, while more capable models are only slightly better at resisting deception. We recommend the development of techniques to detect and defend against deception.