Word2Vec vs DBnary: Augmenting METEOR using Vector Representations or Lexical Resources?
This work addresses the need for better evaluation metrics in machine translation, but it is incremental as it builds upon the existing METEOR framework.
The paper tackled the problem of improving machine translation evaluation by augmenting the METEOR metric with either lexical resources or distributed word representations, showing that vector representations provide a viable alternative and can add useful information.
This paper presents an approach combining lexico-semantic resources and distributed representations of words applied to the evaluation in machine translation (MT). This study is made through the enrichment of a well-known MT evaluation metric: METEOR. This metric enables an approximate match (synonymy or morphological similarity) between an automatic and a reference translation. Our experiments are made in the framework of the Metrics task of WMT 2014. We show that distributed representations are a good alternative to lexico-semantic resources for MT evaluation and they can even bring interesting additional information. The augmented versions of METEOR, using vector representations, are made available on our Github page.