Learning a Formality-Aware Japanese Sentence Representation
This addresses the need for formality-aware sentence representations in downstream tasks like translation for Japanese, enabling appropriate social formality levels, but it is incremental as it builds on existing procedural classification methods.
The paper tackles the problem of preserving formality information in Japanese sentence representations for encoder-decoder models, proposing a method that conditions generation on formality constraints and augments representations with learned formality features, resulting in improved formality recovery and slight semantic preservation gains.
While the way intermediate representations are generated in encoder-decoder sequence-to-sequence models typically allow them to preserve the semantics of the input sentence, input features such as formality might be left out. On the other hand, downstream tasks such as translation would benefit from working with a sentence representation that preserves formality in addition to semantics, so as to generate sentences with the appropriate level of social formality -- the difference between speaking to a friend versus speaking with a supervisor. We propose a sequence-to-sequence method for learning a formality-aware representation for Japanese sentences, where sentence generation is conditioned on both the original representation of the input sentence, and a side constraint which guides the sentence representation towards preserving formality information. Additionally, we propose augmenting the sentence representation with a learned representation of formality which facilitates the extraction of formality in downstream tasks. We address the lack of formality-annotated parallel data by adapting previous works on procedural formality classification of Japanese sentences. Experimental results suggest that our techniques not only helps the decoder recover the formality of the input sentence, but also slightly improves the preservation of input sentence semantics.