From Tokens to Thoughts: How LLMs and Humans Trade Compression for Meaning
This work addresses the problem of understanding differences in knowledge representation between artificial and biological intelligence for AI researchers, offering insights to guide development toward more human-aligned AI, though it is incremental in building on existing categorization benchmarks.
The study applied the Information Bottleneck principle to compare how LLMs and humans balance compression and meaning in knowledge organization, finding that LLMs align broadly with human categories but miss fine-grained distinctions, achieve optimal compression efficiency, and show encoder models outperform decoder models in human alignment.
Humans organize knowledge into compact categories that balance compression with semantic meaning preservation. Large Language Models (LLMs) demonstrate striking linguistic abilities, yet whether they achieve this same balance remains unclear. We apply the Information Bottleneck principle to quantitatively compare how LLMs and humans navigate this compression-meaning trade-off. Analyzing embeddings from 40+ LLMs against classic human categorization benchmarks, we uncover three key findings. First, LLMs broadly align with human categories but miss fine-grained semantic distinctions crucial for human understanding. Second, LLMs demonstrate aggressive statistical compression, achieving ``optimal'' information-theoretic efficiency, while humans prioritize contextual richness and adaptive flexibility. Third, encoder models surprisingly outperform decoder models in human alignment, suggesting that generation and understanding rely on distinct mechanisms in current architectures. In addition, training dynamics analysis reveals that conceptual structure develops in distinct phases: rapid initial formation followed by architectural reorganization, with semantic processing migrating from deeper to mid-network layers as models discover more efficient encoding. These divergent strategies, where LLMs optimize for compression and humans for adaptive utility, reveal fundamental differences between artificial and biological intelligence, guiding development toward more human-aligned AI.