Prospects for inconsistency detection using large language models and sheaves
This work addresses the problem of inconsistency detection in legal and social media contexts, offering a potential tool for improving government consistency and combating misinformation, though it appears incremental as it builds on existing LLM capabilities with a novel mathematical framework.
The paper tackles the problem of detecting logical inconsistencies in texts by demonstrating that large language models can produce reasonable numerical consistency ratings for claims, and proposes a sheaf theory-based mathematical approach to extend these ratings to hypertexts like laws and social media for global consistency evaluation.
We demonstrate that large language models can produce reasonable numerical ratings of the logical consistency of claims. We also outline a mathematical approach based on sheaf theory for lifting such ratings to hypertexts such as laws, jurisprudence, and social media and evaluating their consistency globally. This approach is a promising avenue to increasing consistency in and of government, as well as to combating mis- and disinformation and related ills.