From Circuit Evidence to Mechanistic Theory: An Inductive Logic Approach
For researchers in mechanistic interpretability, this work provides a formal framework to move from isolated circuit findings to cumulative mechanistic science.
The paper introduces a formal infrastructure for mechanistic interpretability by representing circuits as inductive theories, enabling explicit, comparable, and portable mechanistic claims across model scales. Causal Functional Signatures reveal distinct computational strategies, and ILP signatures achieve better structural separation than baselines.
Mechanistic interpretability produces circuit-level causal analyses of neural network behaviour, but discovered circuits often remain isolated experimental artefacts: there is no shared formal representation for what circuits compute, how they relate, or when two findings provide evidence for the same mechanism. This work provides a formal infrastructure for cumulative mechanistic science by treating circuit interpretation as inductive theory construction. Each circuit is characterised at two levels: a Causal Functional Signature (CFS), which grounds component behaviour in causal attribution evidence and token role profiles, and an architectural signature $τ_{\mathrm{arch}}$, learned by inductive logic programming (ILP) from scale-invariant structural predicates. Together, these constitute a formal coherence layer that makes mechanistic claims explicit, comparable via $θ$-subsumption, and portable across model scales. CFS reveals qualitatively distinct computational strategies across task types, including attention-mediated copying versus MLP-mediated binding. ILP signatures achieve substantially better structural separation than graph kernel and feature-vector baselines, and support principled transfer across model scales and architecture families.