Measuring Uncertainty in Transformer Circuits with Effective Information Consistency
This work addresses the need for formal, single-pass uncertainty measurement in mechanistic interpretability for researchers, but it is incremental as it builds on prior systems-theoretic proposals without empirical validation.
The authors tackled the problem of quantifying the coherence and trustworthiness of Transformer Circuits in large language models by introducing the Effective-Information Consistency Score (EICS), which combines sheaf inconsistency and causal emergence measures, but empirical validation on LLM tasks is deferred.
Mechanistic interpretability has identified functional subgraphs within large language models (LLMs), known as Transformer Circuits (TCs), that appear to implement specific algorithms. Yet we lack a formal, single-pass way to quantify when an active circuit is behaving coherently and thus likely trustworthy. Building on prior systems-theoretic proposals, we specialize a sheaf/cohomology and causal emergence perspective to TCs and introduce the Effective-Information Consistency Score (EICS). EICS combines (i) a normalized sheaf inconsistency computed from local Jacobians and activations, with (ii) a Gaussian EI proxy for circuit-level causal emergence derived from the same forward state. The construction is white-box, single-pass, and makes units explicit so that the score is dimensionless. We further provide practical guidance on score interpretation, computational overhead (with fast and exact modes), and a toy sanity-check analysis. Empirical validation on LLM tasks is deferred.