ALIGNS: Unlocking nomological networks in psychological measurement through a large language model
This addresses a foundational problem in psychological measurement validation that affects clinical trials, public policy, and various disciplines, representing a novel application rather than an incremental improvement.
The authors tackled the challenge of building nomological networks for psychological measurement validation, which has remained unresolved for 70 years, by introducing ALIGNS, a large language model-based system that generated three comprehensive networks with over 550,000 indicators across multiple fields and revealed new insights such as anxiety and depression measures converging into a single dimension.
Psychological measurement is critical to many disciplines. Despite advances in measurement, building nomological networks, theoretical maps of how concepts and measures relate to establish validity, remains a challenge 70 years after Cronbach and Meehl proposed them as fundamental to validation. This limitation has practical consequences: clinical trials may fail to detect treatment effects, and public policy may target the wrong outcomes. We introduce Analysis of Latent Indicators to Generate Nomological Structures (ALIGNS), a large language model-based system trained with validated questionnaire measures. ALIGNS provides three comprehensive nomological networks containing over 550,000 indicators across psychology, medicine, social policy, and other fields. This represents the first application of large language models to solve a foundational problem in measurement validation. We report classification accuracy tests used to develop the model, as well as three evaluations. In the first evaluation, the widely used NIH PROMIS anxiety and depression instruments are shown to converge into a single dimension of emotional distress. The second evaluation examines child temperament measures and identifies four potential dimensions not captured by current frameworks, and questions one existing dimension. The third evaluation, an applicability check, engages expert psychometricians who assess the system's importance, accessibility, and suitability. ALIGNS is freely available at nomologicalnetwork.org, complementing traditional validation methods with large-scale nomological analysis.