SELF-[IN]CORRECT: LLMs Struggle with Discriminating Self-Generated Responses
This addresses a fundamental limitation in LLM self-improvement for AI researchers and practitioners, showing it is incremental by questioning a common assumption.
The paper tackled the problem of whether LLMs can consistently improve their outputs by discriminating among self-generated alternatives, finding that models are not reliably better at discrimination than initial generation, challenging the idea of self-enhancement.
Can LLMs consistently improve their previous outputs for better results? For this to be true, LLMs would need to be better at discriminating among previously-generated alternatives, than generating initial responses. We explore the validity of this hypothesis in practice. We first formulate a unified framework that allows us to compare the generative and discriminative capability of any model on any task. In our resulting experimental analysis of several open-source and industrial LLMs, we observe that models are not reliably better at discriminating among previously-generated alternatives than generating initial responses. This finding challenges the notion that LLMs may be able to enhance their performance only through their own judgment.