Pavel Levin

Emmanouil Stergiadis, Satendra Kumar, Fedor Kovalev et al.

While NMT has achieved remarkable results in the last 5 years, production systems come with strict quality requirements in arbitrarily niche domains that are not always adequately covered by readily available parallel corpora. This is typically addressed by training domain specific models, using fine-tuning methods and some variation of back-translation on top of in-domain monolingual corpora. However, industrial practitioners can rarely afford to focus on a single domain. A far more typical scenario includes a set of closely related, yet succinctly different sub-domains. At Booking.com, we need to translate property descriptions, user reviews, as well as messages, (for example those sent between a customer and an agent or property manager). An editor might need to translate articles across a set of different topics. An e-commerce platform would typically need to translate both the description of each item and the user generated content related to them. To this end, we propose MDT: a novel method to simultaneously fine-tune on several sub-domains by passing multidimensional sentence-level information to the model during training and inference. We show that MDT achieves results competitive to N specialist models each fine-tuned on a single constituent domain, while effectively serving all N sub-domains, therefore cutting development and maintenance costs by the same factor. Besides BLEU (industry standard automatic evaluation metric known to only weakly correlate with human judgement) we also report rigorous human evaluation results for all models and sub-domains as well as specific examples that better contextualise the performance of each model in terms of adequacy and fluency. To facilitate further research, we plan to make the code available upon acceptance.

Benjamin Gutelman, Pavel Levin

Image galleries provide a rich source of diverse information about a product which can be leveraged across many recommendation and retrieval applications. We study the problem of building a universal image gallery encoder through multi-task learning (MTL) approach and demonstrate that it is indeed a practical way to achieve generalizability of learned representations to new downstream tasks. Additionally, we analyze the relative predictive performance of MTL-trained solutions against optimal and substantially more expensive solutions, and find signals that MTL can be a useful mechanism to address sparsity in low-resource binary tasks.

Pavel Levin, Nishikant Dhanuka, Talaat Khalil et al.

While some remarkable progress has been made in neural machine translation (NMT) research, there have not been many reports on its development and evaluation in practice. This paper tries to fill this gap by presenting some of our findings from building an in-house travel domain NMT system in a large scale E-commerce setting. The three major topics that we cover are optimization and training (including different optimization strategies and corpus sizes), handling real-world content and evaluating results.

Pavel Levin, Nishikant Dhanuka, Maxim Khalilov

We describe our recently developed neural machine translation (NMT) system and benchmark it against our own statistical machine translation (SMT) system as well as two other general purpose online engines (statistical and neural). We present automatic and human evaluation results of the translation output provided by each system. We also analyze the effect of sentence length on the quality of output for SMT and NMT systems.