Navigating the Conceptual Multiverse
For users of open-ended language models, this work provides a novel method to make hidden decision spaces transparent and verifiable, improving interpretability and control.
The paper introduces the conceptual multiverse, an interactive system that maps hidden decisions in language model outputs, enabling users to inspect, modify, and verify them against domain reasoning. Across three domains, it improved user understanding and output quality, e.g., philosophy students rewrote essays with sharper framings and reversed theses.
When language models answer open-ended problems, they implicitly make hidden decisions that shape their outputs, leaving users with uncontextualized answers rather than a working map of the problem; drawing on multiverse analysis from statistics, we build and evaluate the conceptual multiverse, an interactive system that represents conceptual decisions such as how to frame a question or what to value as a space users can transparently inspect, intervenably change, and check against principled domain reasoning; for this structure to be worth navigating rather than misleading, it must be rigorous and checkable against domain reasoning norms, so we develop a general verification framework that enforces properties of good decision structures like unambiguity and completeness calibrated by expert-level reasoning; across three domains, the conceptual multiverse helped participants develop a working map of the problem, with philosophy students rewriting essays with sharper framings and reversed theses, alignment annotators moving from surface preferences to reasoning about user intent and harm, and poets identifying compositional patterns that clarified their taste.