Learning Disentangled Representations of Negation and Uncertainty
This work addresses a specific challenge in NLP for tasks requiring precise semantic understanding, but it is incremental as it builds on existing representation learning approaches.
The paper tackled the problem of modeling negation and uncertainty in natural language processing by proposing a method to disentangle their representations from content using a Variational Autoencoder, finding that supervision combined with adversarial learning and mutual information minimization improved disentanglement.
Negation and uncertainty modeling are long-standing tasks in natural language processing. Linguistic theory postulates that expressions of negation and uncertainty are semantically independent from each other and the content they modify. However, previous works on representation learning do not explicitly model this independence. We therefore attempt to disentangle the representations of negation, uncertainty, and content using a Variational Autoencoder. We find that simply supervising the latent representations results in good disentanglement, but auxiliary objectives based on adversarial learning and mutual information minimization can provide additional disentanglement gains.