Implicit In-Context Learning: Evidence from Artificial Language Experiments
This addresses the problem of understanding the cognitive mechanisms of LLMs for researchers in AI and linguistics, but it is incremental as it builds on existing artificial language experiments.
The study investigated whether large language models exhibit human-like implicit pattern recognition during in-context learning by testing them on artificial language tasks in morphology, morphosyntax, and syntax. Results showed domain-specific alignment with human behaviors, with o3-mini aligning better in morphology and both models aligning in syntax.
Humans acquire language through implicit learning, absorbing complex patterns without explicit awareness. While LLMs demonstrate impressive linguistic capabilities, it remains unclear whether they exhibit human-like pattern recognition during in-context learning at inferencing level. We adapted three classic artificial language learning experiments spanning morphology, morphosyntax, and syntax to systematically evaluate implicit learning at inferencing level in two state-of-the-art OpenAI models: gpt-4o and o3-mini. Our results reveal linguistic domain-specific alignment between models and human behaviors, o3-mini aligns better in morphology while both models align in syntax.