Mining Logical Event Schemas From Pre-Trained Language Models
This work addresses the problem of automated schema learning for everyday scenarios, which could benefit natural language understanding applications, though it appears incremental as it builds on existing methods like FrameNet and language models.
The authors tackled the problem of learning event schemas from language by developing NESL, a system that extracts situation samples from pre-trained language models, parses them into FrameNet frames, and generalizes them into hierarchical schemas. They showed that careful sampling emphasizes stereotypical properties and results in schemas that specify situations more comprehensively than other systems.
We present NESL (the Neuro-Episodic Schema Learner), an event schema learning system that combines large language models, FrameNet parsing, a powerful logical representation of language, and a set of simple behavioral schemas meant to bootstrap the learning process. In lieu of a pre-made corpus of stories, our dataset is a continuous feed of "situation samples" from a pre-trained language model, which are then parsed into FrameNet frames, mapped into simple behavioral schemas, and combined and generalized into complex, hierarchical schemas for a variety of everyday scenarios. We show that careful sampling from the language model can help emphasize stereotypical properties of situations and de-emphasize irrelevant details, and that the resulting schemas specify situations more comprehensively than those learned by other systems.