Daniel Klein

Zhiheng Lyu, Kevin Yang, Lingpeng Kong et al.

While accurately detecting and correcting factual contradictions in language model outputs has become increasingly important as their capabilities improve, doing so is highly challenging. We propose a novel method, FACTTRACK, for tracking atomic facts and addressing factual contradictions. Crucially, FACTTRACK also maintains time-aware validity intervals for each fact, allowing for change over time. At a high level, FACTTRACK consists of a four-step pipeline to update a world state data structure for each new event: (1) decompose the event into directional atomic facts; (2) determine the validity interval of each atomic fact using the world state; (3) detect contradictions with existing facts in the world state; and finally (4) add new facts to the world state and update existing atomic facts. When we apply FACTTRACK to contradiction detection on structured story outlines, we find that FACTTRACK using LLaMA2-7B-Chat substantially outperforms a fair baseline using LLaMA2-7B-Chat, and achieves performance comparable to a GPT4 baseline. Moreover, when using GPT4, FACTTRACK significantly outperforms the GPT4 baseline.

Atiye Alaeddini, Daniel Klein

Decisions in public health are almost always made in the context of uncertainty. Policy makers are responsible for making important decisions, faced with the daunting task of choosing from amongst many possible options. This task is called planning under uncertainty, and is particularly acute when addressing complex systems, such as issues of global health and development. Uncertainty leads to cautious or incorrect decisions that cost time, money, and human life. It is with this understanding that we pursue greater clarity on, and methods to address optimal policy making in health. Decision making under uncertainty is a challenging task, and all too often this uncertainty is averaged away to simplify results for policy makers. Our goal in this work is to implement dynamic programming which provides basis for compiling planning results into reactive strategies. We present here a description of an AI-based method and illustrate how this method can improve our ability to find an optimal vaccination strategy. We model the problem as a partially observable Markov decision process, POMDP and show how a re-active policy can be computed using dynamic programming. In this paper, we developed a framework for optimal health policy design in an uncertain dynamic setting. We apply a stochastic dynamic programming approach to identify the optimal time to change the health intervention policy and the value of decision relevant information for improving the impact of the policy.