CoRE: Condition-based Reasoning for Identifying Outcome Variance in Complex Events
This work addresses the challenge of critically examining claims about complex event outcomes for researchers and practitioners, but it is incremental as it builds on existing datasets and methods.
The paper tackled the problem of identifying which latent conditions lead to specific outcomes in complex events, finding that conditions are useful when context is incomplete and that models vary widely in their ability to generate and identify outcome-variant conditions, with larger models like GPT-4o being more cautious in less constrained situations.
Knowing which latent conditions lead to a particular outcome is useful for critically examining claims made about complex event outcomes. Identifying implied conditions and examining their influence on an outcome is challenging. We handle this by combining and augmenting annotations from two existing datasets consisting of goals and states, and explore the influence of conditions through our research questions and Condition-based Reasoning tasks. We examine open and closed LLMs of varying sizes and intent-alignment on our reasoning tasks and find that conditions are useful when not all context is available. Models differ widely in their ability to generate and identify outcome-variant conditions which affects their performance on outcome validation when conditions are used to replace missing context. Larger models like GPT-4o, are more cautious in such less constrained situations.