ExplicitLM: Decoupling Knowledge from Parameters via Explicit Memory Banks
This addresses the issue of opaque and hard-to-update knowledge in AI models for researchers and practitioners, offering a novel approach with interpretability and updatability, though it builds on existing ideas like memory networks and RAG.
The paper tackles the problem of knowledge staleness and lack of interpretability in large language models by proposing ExplicitLM, an architecture with an external memory bank for explicit knowledge storage, achieving up to 43.67% improvement on knowledge-intensive tasks and 3.62× gains in low-data regimes.
Large language models suffer from knowledge staleness and lack of interpretability due to implicit knowledge storage across entangled network parameters, preventing targeted updates and reasoning transparency. We propose ExplicitLM, a novel architecture featuring a million-scale external memory bank storing human-readable knowledge as token sequences, enabling direct inspection and modification. We design a differentiable two-stage retrieval mechanism with efficient coarse-grained filtering via product key decomposition (reducing complexity from $\mathcal{O}(N \cdot |I|)$ to $\mathcal{O}(\sqrt{N} \cdot |I|)$) and fine-grained Gumbel-Softmax matching for end-to-end training. Inspired by dual-system cognitive theory, we partition knowledge into frozen explicit facts (20%) and learnable implicit patterns (80%), maintained through Exponential Moving Average updates for stability. ExplicitLM achieves up to 43.67% improvement on knowledge-intensive tasks versus standard Transformers, with 3.62$\times$ gains in low-data regimes (10k samples). Analysis shows strong correlations between memory retrieval and performance, with correct predictions achieving 49% higher hit rates. Unlike RAG systems with frozen retrieval, our jointly optimized architecture demonstrates that interpretable, updatable models can maintain competitive performance while providing unprecedented knowledge transparency.