This conference builds on a series of annual workshops and conferences on statistical machine
translation, going back to 2006:
|Release of training data for shared tasks||January/February, 2017|
|Evaluation periods for shared tasks||April/May, 2017|
|Conference in Copenhagen||September 7-8, 2017|
This year's conference will feature the following shared tasks:
- a news translation task,
- a biomedical translation task ,
- an automatic post-editing task,
- a metrics task (assess MT quality given reference translation).
- a quality estimation task (assess MT quality without access to any reference),
- a multimodal translation task
- a task dedicated to the training of neural MT systems
In addition to the shared tasks, the conference will also feature scientific papers on topics related to MT.
Topics of interest include, but are not limited to:
We encourage authors to evaluate their approaches to the above topics
using the common data sets created for the shared tasks.
- word-based, phrase-based, syntax-based, semantics-based SMT
- neural machine translation
- using comparable corpora for SMT
- incorporating linguistic information into SMT
- system combination
- error analysis
- manual and automatic method for evaluating MT
- scaling MT to very large data sets
NEWS TRANSLATION TASK
The first shared task which will examine translation between the
following language pairs:
The text for all the test sets will be drawn from news articles.
Participants may submit translations for any or all of the language
directions. In addition to the common test sets the conference organizers
will provide optional training resources.
- English-Chinese and Chinese-English NEW
- English-Czech and Czech-English
- English-Finnish and Finnish-English
- English-German and German-English
- English-Latvian and Latvian-English NEW
- English-Russian and Russian-English
- English-Turkish and Turkish-English
All participants who submit entries will have their translations
evaluated. We will evaluate translation performance by human judgment. To
facilitate the human evaluation we will require participants in the
shared tasks to manually judge some of the submitted translations. For each team,
this will amount
to ranking 300 sets of 5 translations, per language pair submitted.
BIOMEDICAL TRANSLATION TASK
In this second edition of this task, we will evaluate systems for the translation of
clinical trials and scientific abstracts in biological and health sciences for the following languages pairs:
- English-French and French-English
- English-Spanish and Spanish-English
- English-Portuguese and Portuguese-English
- English-German and German-English NEW
- among others...
Parallel corpora will be available for all language pairs but also monoligual corpora for each of the languages.
Evaluation will be carried out both automatically and manually.
The metrics task (also called evaluation task) will assess automatic evaluation metrics' ability to:
- Evaluate systems on their overall performance on the test set
- Evaluate systems on a sentence by sentence level
Participants in the shared evaluation task will use their automatic evaluation metrics to score the output from the translation task and the NMT training task. In addition to MT outputs from the other two tasks, the participants will be provided with reference translations. We will measure the correlation of automatic evaluation metrics with the human judgments.
NEURAL MT TRAINING TASK (NMT TRAINING TASK)
This task will assess your team's ability to train a fixed neural MT model given fixed data.
Participants in the NMT training task will be given complete a Neural Monkey configuration file which describes the neural model. Training and validation data with fixed pre-processing scheme will be also given (English-to-Czech and Czech-to-English translation).
The participants will be expected to submit the variables file, i.e. the trained neural network, for one or both of the translation directions. We will use the variables and a fixed revision of Neural Monkey to translate official WMT17 test set. The outputs of the various configurations of the system will be scored using the standard manual evaluation procedure.
Subscribe to to the announcement list for WMT by entering your e-mail address below. This list will be used to announce when the test sets are released, to indicate any corrections to the training sets, and to amend the deadlines as needed.
You can read past
announcements on the Google Groups page for WMT. These also
include an archive of announcements from earlier workshops.||
Ondřej Bojar (Charles University in Prague)
Christian Buck (University of Edinburgh)
Rajen Chatterjee (FBK)
Christian Federmann (MSR)
Barry Haddow (University of Edinburgh)
Matthias Huck (University of Edinburgh)
Antonio Jimeno Yepes (IBM Research Australia)
Julia Kreutzer (Heidelberg University)
Aurélie Névéol (LIMSI, CNRS)
Mariana Neves (Hasso-Plattner Institute)
Philipp Koehn (University of Edinburgh / Johns Hopkins University)
Christof Monz (University of Amsterdam)
Matteo Negri (FBK)
Matt Post (Johns Hopkins University)
Stefan Riezler (Heidelberg University)
Artem Sokolov (Heidelberg University, Amazon Development Center, Berlin)
Lucia Specia (University of Sheffield)
Karin Verspoor (University of Melbourne)
Marco Turchi (FBK)
For general questions, comments, etc. please send email
For task-specific questions, please contact the relevant organisers.
This conference has received funding
from the European Union’s Horizon 2020 research
and innovation programme under grant agreements
645452 (QT21) and 645357 (Cracker).
We thank Yandex for their donation of data for the Russian-English and Turkish-English news tasks, and the University of Helsinki for their donation for the Finnish-English news tasks.