PHALM: Building a Knowledge Graph from Scratch by Prompting Humans and a Language Model
This addresses the challenge of obtaining commonsense knowledge for natural language understanding models, particularly for Japanese, though it is incremental as it builds on existing prompting and knowledge graph techniques.
The authors tackled the problem of building a high-quality knowledge graph from scratch at low cost by proposing PHALM, a method that prompts both crowdworkers and a large language model, and used it to create a Japanese event knowledge graph and train commonsense generation models, with experimental results showing acceptability of the graph and inferences.
Despite the remarkable progress in natural language understanding with pretrained Transformers, neural language models often do not handle commonsense knowledge well. Toward commonsense-aware models, there have been attempts to obtain knowledge, ranging from automatic acquisition to crowdsourcing. However, it is difficult to obtain a high-quality knowledge base at a low cost, especially from scratch. In this paper, we propose PHALM, a method of building a knowledge graph from scratch, by prompting both crowdworkers and a large language model (LLM). We used this method to build a Japanese event knowledge graph and trained Japanese commonsense generation models. Experimental results revealed the acceptability of the built graph and inferences generated by the trained models. We also report the difference in prompting humans and an LLM. Our code, data, and models are available at github.com/nlp-waseda/comet-atomic-ja.