Pitfalls of Scale: Investigating the Inverse Task of Redefinition in Large Language Models
This work addresses potential reasoning gaps in LLMs for AI safety and reliability, though it is incremental in exploring scaling effects.
The study investigated how large language models (LLMs) handle redefinition tasks, where physical constants and units are assigned alternative values, finding that performance degrades and false confidence increases with model scale, despite factors like prompting strategies.
Inverse tasks can uncover potential reasoning gaps as Large Language Models (LLMs) scale up. In this work, we explore the redefinition task, in which we assign alternative values to well-known physical constants and units of measure, prompting LLMs to respond accordingly. Our findings show that not only does model performance degrade with scale, but its false confidence also rises. Moreover, while factors such as prompting strategies or response formatting are influential, they do not preclude LLMs from anchoring to memorized values.