Human-Inspired Learning for Large Language Models via Obvious Record and Maximum-Entropy Method Discovery
This addresses the issue of LLMs' limited ability to handle low-resource cases for applications like niche hardware or IoT, though it appears incremental as it builds on existing learning paradigms.
The paper tackles the problem of LLMs struggling with rare or unseen scenarios by proposing a human-inspired learning framework that integrates Obvious Record for explicit symbolic memory and Maximum-Entropy Method Discovery for diverse strategy capture, achieving stronger coverage and greater internal diversity on a benchmark of 60 question-solution pairs.
Large Language Models (LLMs) excel at extracting common patterns from large-scale corpora, yet they struggle with rare, low-resource, or previously unseen scenarios-such as niche hardware deployment issues or irregular IoT device behaviors-because such cases are sparsely represented in training data. Moreover, LLMs rely primarily on implicit parametric memory, which limits their ability to explicitly acquire, recall, and refine methods, causing them to behave predominantly as intuition-driven predictors rather than deliberate, method-oriented learners. Inspired by how humans learn from rare experiences, this paper proposes a human-inspired learning framework that integrates two complementary mechanisms. The first, Obvious Record, explicitly stores cause--result (or question--solution) relationships as symbolic memory, enabling persistent learning even from single or infrequent encounters. The second, Maximum-Entropy Method Discovery, prioritizes and preserves methods with high semantic dissimilarity, allowing the system to capture diverse and underrepresented strategies that are typically overlooked by next-token prediction. Verification on a benchmark of 60 semantically diverse question--solution pairs demonstrates that the proposed entropy-guided approach achieves stronger coverage of unseen questions and significantly greater internal diversity than a random baseline, confirming its effectiveness in discovering more generalizable and human-inspired methods.