Let Me Teach You: Pedagogical Foundations of Feedback for Language Models
This work addresses the problem of arbitrary feedback design in NLF for researchers and practitioners, offering a foundational framework to guide future work, though it is incremental in applying established pedagogical ideas to this domain.
The paper tackles the lack of systematic grounding in Natural Language Feedback (NLF) methods for aligning Large Language Models to human preferences by introducing FELT, a feedback framework based on pedagogical principles that outlines feedback characteristics and a content taxonomy to streamline designs and identify new research directions.
Natural Language Feedback (NLF) is an increasingly popular mechanism for aligning Large Language Models (LLMs) to human preferences. Despite the diversity of the information it can convey, NLF methods are often hand-designed and arbitrary, with little systematic grounding. At the same time, research in learning sciences has long established several effective feedback models. In this opinion piece, we compile ideas from pedagogy to introduce FELT, a feedback framework for LLMs that outlines various characteristics of the feedback space, and a feedback content taxonomy based on these variables, providing a general mapping of the feedback space. In addition to streamlining NLF designs, FELT also brings out new, unexplored directions for research in NLF. We make our taxonomy available to the community, providing guides and examples for mapping our categorizations to future research.