Gaussian Attention Model and Its Application to Knowledge Base Embedding and Question Answering
This work addresses knowledge base completion and question answering tasks, offering a method to handle uncertainty propagation and conjunction in queries, but it is incremental as it builds on existing attention mechanisms with a specific adaptation.
The authors tackled the problem of knowledge base embedding and question answering by proposing a Gaussian attention model that allows neural networks to adjust attention focus from sharp to broad, enabling handling of path and conjunctive queries. They demonstrated its effectiveness on a dataset of FIFA World Cup 2014 soccer players, showing it can manage both query types well.
We propose the Gaussian attention model for content-based neural memory access. With the proposed attention model, a neural network has the additional degree of freedom to control the focus of its attention from a laser sharp attention to a broad attention. It is applicable whenever we can assume that the distance in the latent space reflects some notion of semantics. We use the proposed attention model as a scoring function for the embedding of a knowledge base into a continuous vector space and then train a model that performs question answering about the entities in the knowledge base. The proposed attention model can handle both the propagation of uncertainty when following a series of relations and also the conjunction of conditions in a natural way. On a dataset of soccer players who participated in the FIFA World Cup 2014, we demonstrate that our model can handle both path queries and conjunctive queries well.