Task description

This task aims to evaluate systems on the translation of documents from the biomedical domain. The test data will consist of Medline abstracts and other biomedical documents. This year, the biomedical translation task will address the following language pairs:

• English-Chinese and Chinese-English
• English-French and French-English
• English-German and German-English
• English-Portuguese and Portuguese-English
• English-Spanish and Spanish-English

Data

• The Biomedical Translation repository includes scientific publications (en/pt, en/es, en/fr) and clinical trials (en/pt).
• The Medline abstracts training data for es/en, de/en, pt/en and fr/en. NEW
• The UFAL Medical Corpus (formerly HimLCorpus) includes medical text from various sources for many language pairs (en/es, en/de, en/fr, en/ro). HimL test sets can be used as the development sets for some language pairs (en/es, en/de, en/fr, en/ro).
• The Khresmoi development data can be used for some language pairs (en/es, en/de, en/fr).
• The UNCorpus contains training data for some languages (en/es, en/fr, en/zh).
• The MeSpEn corpus contains many parallel documents of en/es.
• The Scielo full text corpus for en, es and pt.
• The Brazilian Thesis and Dissertations for en/pt.

Evaluation

• Scientific abstracts from Medline:
• Chinese/English (both directions)
• French/English (both directions)
• Portuguese/English (both directions)
• Spanish/English (both directions)
• German/English (both directions)
• Translation of terms from biomedical terminologies:
• English to Spanish

Test Sets and Submission formats

Scientific abstracts from Medline:

For the test set of Medline abstracts, the format will be plain text files. The format will be the following:

DOC_ID	SENT_ID	SENTENCE_TEXT

• DOC_ID: sequential one, e.g. doc1, doc8, not the original PMID in Medline
• SENT_ID: a sequential number from 1 to n
• SENTENCE_TEXT: the sentence text to be translated by the participants

doc1	1	sentence_1
doc2	2	sentence_2
doc2	3	sentence_3
doc2	4	sentence_4
doc2	5	sentence_5
...
doc2	n	sentence_n
doc4	1	sentence_1
doc4	2	sentence_2
...

doc1	1	translated_sentence_1
doc2	2	translated_sentence_2
doc2	3	translated_sentence_3
doc2	4	translated_sentence_4
doc2	5	translated_sentence_5
...
doc2	n	translated_sentence_n
doc4	1	translated_sentence_1
doc4	2	translated_sentence_2
...

Terms from biomedical terminologies:

For the translation of terms, the format of the test set will be one term per line, as in the example below:

Coronal craniosynostosis
growth hormone treatment
Stenosis of right renal artery
17q21 Microdeletion Syndrome
Paraplegia/paraparesis
Mitochondrial pathology
acute aortic dissection
Glaucoma of left eye
Otitis media of left ear
Renal cortical atrophy

Craneosinostosis coronal
tratamiento con hormona del crecimiento
Estenosis de la arteria renal derecha
Síndrome de microdeleción 17q21
Paraplejia/paraparesia
Enfermedad mitocondrial
disección aórtica aguda
Glaucoma del ojo izquierdo
Otitis media del oído izquierdo
Atrofia cortical renal

Submission Requirements

Please register your team using this form. You will receive a mail with the confirmation of your registration. The link for submission site will be informed in this mail Please register your team as soon as possible.

The test files will be available in the WMT'19 biomedical task Google Drive folder.

The format for the submission files should include the original test file name preceded by the team identifier (as registered in the form above) and the run number, following this example:

• The submission file for run 1 of the "ABC" team for the Medline dataset for English to Spanish should be called "ABC_run1_medline_en2es_es.txt".
• The submission file for run 3 of the "ABC" team for the Medline dataset for Spanish to English should be called "ABC_run3_medline_es2en_en.txt".

• The submission file for run 1 of the "ABC" team for the terminology dataset should be called "ABC_run1_terms.txt".

Each team will be allowed to submit up to 3 runs per test set.

Results

Results for the biomedical task are available here.

• Ten "medline_*" files with the reference sentences for each Medline test set.
• Five "mapdocs_*" files for each language pair for mapping document sequentials to PubMed PMIDs.
• Five "align_validation_*" files for each language pair with the manual validation of the automatic alignment. The order of the languages in the alignment (column 2 in the TAB-separated file) is foreign language vs. (<=>) English.

Important dates

Organisers