Shared Task: Similar Language Translation

TASK DESCRIPTION

Within the MT and NLP communities, English is by far the most resource-rich language. MT systems are most often trained to translate texts from and to English or they use English as a pivot language to translate between resource-poorer languages. The interest in English is reflected, for example, in the WMT translation tasks (e.g. News, Biomedical) which have always included language pairs in which texts are translated to and/or from English.

With the widespread use of MT technology, there is more and more interest in training systems to translate between languages other than English. One evidence of this is the need of directly translating between pairs of similar languages. The main challenge here is how to take advantage of the similarity between languages to overcome the limitation given the low amount of available parallel data to produce an accurate output.

Given the interest of the community in this topic we organize, for the first time at WMT, a shared task on "Similar Language Translation" to evaluate the performance of state-of-the-art translation systems on translating between pairs of languages from the same language family. We provide participants with training and testing data from three language pairs: Spanish - Portuguese (Romance languages), Czech - Polish (Slavic languages), and Hindi - Nepali (Indo-Aryan languages). Evaluation will be carried out using automatic evaluation metrics and human evaluation.

Language Pairs

Utilizing parallel data

No additional parallel data is allowed for training. Constrained submissions only.

Utilizing monolingual data

You are encouraged to develop novel solutions to utilize monolingual corpora to improve translation quality.

DATA

The training and dev sets are available here.

To receive the test data, available on April 10, please fill out the registration form.

EVALUATION

The evaluation has been carried out automatically using BLEU (Papieni et al., 2002) and TER (Snover et al., 2006).

All systems are ranked by BLEU score. TER scores have been calculated for systems which obtained BLEU scores greater than 5.0.

The official results of the competition are available here.

IMPORTANT DATES

Release of training/dev data February 24, 2019
Test data released April 15, 2019
Submission deadline April 24, 2019
System description paper deadline May 20, 2019
Notifications June 10, 2019
Camera-ready June 17, 2019

ORGANIZERS

CONTACT

martaruizcostajussa(at)gmail.com

ACKNOWLEDGEMENT

We would like to thank AT Language Solutions for providing us with part of the data used in this task.