Evaluating Byte and Wordpiece Level Models for Massively Multilingual Semantic Parsing
This work addresses the challenge of multilingual semantic parsing for NLP researchers, but it is incremental as it compares existing models on a specific dataset.
The paper tackled the problem of comparing byte-level (ByT5) and wordpiece-based (mT5) models for multilingual semantic parsing across 51 languages, finding that using synthetic data with label projection reduced the accuracy gap to only 5 points compared to training on full gold data.
Token free approaches have been successfully applied to a series of word and span level tasks. In this work, we compare a byte-level (ByT5) and a wordpiece based (mT5) sequence to sequence model on the 51 languages of the MASSIVE multilingual semantic parsing dataset. We examine multiple experimental settings: (i) zero-shot, (ii) full gold data and (iii) zero-shot with synthetic data. By leveraging a state-of-the-art label projection method for machine translated examples, we are able to reduce the gap in exact match accuracy to only 5 points with respect to a model trained on gold data from all the languages. We additionally provide insights on the cross-lingual transfer of ByT5 and show how the model compares with respect to mT5 across all parameter sizes.