Sushrut Thorat

Aadi Narayana Varma Dantuluri, Sushrut Thorat, Paras Chopra

Artificial intelligence systems increasingly mediate how science is found and credited. We asked whether missing metadata prevents AI systems from crediting work. As a boundary test, an AI system citing without access to task-relevant paper lists often produced out-of-list identifiers, some fabricated. We then tested the mechanism in real scholarly infrastructure by using OpenAlex records to hide or restore author, institution, funder, reference, and text-access links while holding works and tasks fixed. Restoring the relevant link made the corresponding attribution possible; restoring the wrong kind did not, with 0 correct answers across 469 completed mismatched tests. Thus, in these tasks, one metadata facet did not substitute for another. Missing links led to invented answers, refusals, or tool-budget exhaustion, and web search did not recover hidden author links. In sum, AI systems credited work only when record connections were visible or recoverable. This motivates Nexus-Score, a record-level check for metadata gaps, to guide repair and help prepare the scholarly record for AI-mediated use.

Ishita Darade, Sushrut Thorat

Structured prompts require integrating components according to task-relevant relations. How a network implements this integration is often hard to judge in language or vision, where those relations are rarely specified precisely enough to define a candidate internal algorithm. Arithmetic offers a cleaner setting. We study a Transformer trained on base-digit extraction: given $N$, $B$, and $D$, it must report the coefficient of $B^D$ in the base-$B$ expansion of $N$. The closed-form solution, $\lfloor N/B^D \rfloor \bmod B$, provides explicit candidate algorithmic intermediates. Across three seeds, the model reaches 99.83% exact-answer accuracy on held-out number-base intersections, establishing reliable task competence. Linear probes decode the intermediates, making staged arithmetic computation plausible. Causal tests then separate representation from use: within the localized route from the stream with $D$ as input to the output positions, behavior depends on early $D$-selective communication, independent of $N$ and $B$. Relatedly, a sparse circuit search finds mostly separate $N$, $B$, and $D$ routes that combine late rather than the staged route suggested by the probes. Thus, the model represents the intermediates that make the closed-form solution plausible, but the identified localized causal route does not transmit them to the output stream. This case shows that probe-based conclusions can diverge sharply from causal observations, even when explicit algorithmic hypotheses are available.