EvEntS ReaLM: Event Reasoning of Entity States via Language Models
This work addresses the challenge of improving event reasoning in AI systems for applications like robotics or natural language understanding, though it is incremental as it builds on existing prompting methods.
The paper tackles the problem of predicting entity state-changes in event reasoning using language models, finding that while LLMs often fail at this task, proper prompting techniques can significantly improve performance, especially for out-of-domain attributes or limited data scenarios.
This paper investigates models of event implications. Specifically, how well models predict entity state-changes, by targeting their understanding of physical attributes. Nominally, Large Language models (LLM) have been exposed to procedural knowledge about how objects interact, yet our benchmarking shows they fail to reason about the world. Conversely, we also demonstrate that existing approaches often misrepresent the surprising abilities of LLMs via improper task encodings and that proper model prompting can dramatically improve performance of reported baseline results across multiple tasks. In particular, our results indicate that our prompting technique is especially useful for unseen attributes (out-of-domain) or when only limited data is available.