Dynamic Meta-Metrics: Source-Sentence Conditioned Weighting for MT Evaluation
For MT evaluation researchers, DMM provides a novel adaptive weighting approach that yields consistent gains over static ensembles, though improvements are incremental.
Dynamic Meta-Metrics (DMM) improves machine translation evaluation by learning source-sentence conditioned combinations of existing metrics, outperforming static ensembles and linear models across multiple language pairs on WMT Metrics Shared Task data.
We propose Dynamic Meta-Metrics (DMM), a framework for machine translation evaluation that learns source-sentence conditioned combinations of existing metrics. Rather than relying on a single static ensemble or language-specific weighting, DMM adapts the metric combination based on properties of the source segment. We study hard conditioning, which fits an interpretable combiner per cluster, and an exploratory soft-conditioned extension whose weights vary continuously with source-cluster responsibilities. We evaluate DMM on the WMT Metrics Shared Task data across multiple language pairs using pairwise agreement measures at the system and segment levels. Across settings, MLP-based combinations outperform linear and Gaussian process-based ensembles, and introducing soft conditioning yields gains over linear models.