Structural Enforcement of Statistical Rigor in AI-Driven Discovery: A Functional Architecture
This addresses the risk of methodological errors in AI-driven discovery for automated science, though it is incremental as it builds on existing functional programming and statistical protocols.
The paper tackles the problem of spurious discoveries in LLM-driven automated research systems by introducing a functional architecture that enforces statistical rigor, validated through large-scale simulation (N=2000 hypotheses) and a case study to ensure science integrity.
Sequential statistical protocols require meticulous state management and robust error handling -- challenges naturally suited to functional programming. We present a functional architecture for structural enforcement of statistical rigor in automated research systems (AI-Scientists). These LLM-driven systems risk generating spurious discoveries through dynamic hypothesis testing. We introduce the Research monad, a Haskell eDSL that enforces sequential statistical protocols (e.g., Online FDR (false discovery rate) control) using a monad transformer stack. To address risks in hybrid architectures where LLMs generate imperative code, we employ Declarative Scaffolding -- generating rigid harnesses that structurally constrain execution and prevent methodological errors like data leakage. We validate this approach through large-scale simulation (N=2000 hypotheses) and an end-to-end case study, demonstrating essential defense-in-depth for automated science integrity.