Iterative Inference in a Chess-Playing Neural Network
This work addresses the internal computational dynamics of neural networks, specifically in chess-playing AI, but is incremental as it extends existing analysis methods to a new domain.
The study investigated whether neural networks refine representations smoothly or through complex processes by analyzing Leela Chess Zero's policy network, finding that while playing strength improved monotonically across layers, policy distributions often followed non-smooth trajectories with early correct solutions being discarded.
Do neural networks build their representations through smooth, gradual refinement, or via more complex computational processes? We investigate this by extending the logit lens to analyze the policy network of Leela Chess Zero, a superhuman chess engine. We find strong monotonic trends in playing strength and puzzle-solving ability across layers, yet policy distributions frequently follow non-smooth trajectories. Evidence for this includes correct puzzle solutions that are discovered early but subsequently discarded, move rankings that remain poorly correlated with final outputs, and high policy divergence until late in the network. These findings contrast with the smooth distributional convergence typically observed in language models.