Multi-Lingual, Multi-Modal, Multi-Task
Training machine translation for multiple language pairs leads to more generalization in the models, and helps low-resource language pairs. Moreover, the input to machine translation may also be enriched by information from other modalities, such as images or speech. And finally, machine translation may just be one task of an integrated neural network that performs other language processing tasks.
Multilingual Multimodal Multitask is the main subject of 11 publications.
Johnson et al. (2016)
explore how well a single canonical neural translation model is able to learn from multiple to multiple languages, by simultaneously training on on parallel corpora for several language pairs. They show small benefits for several input languages with the same output languages, mixed results for translating into multiple output languages (indicated by an additional input language token). The most interesting result is the ability for such a model to translate in language directions for which no parallel corpus is provided, thus demonstrating that some interlingual meaning representation is learned, although less well than using traditional pivot methods.
Firat et al. (2016)
support multi-language input and output by training language-specific encoders and decoders and a shared attention mechanism.
Niehues and Cho (2017)
tackle multiple tasks (translation, part-of-speech tagging, and named entity identification) with shared components of a sequence to sequence model, showing that training on several tasks improves performance on each individual task.
- Delbrouck and Dupont (2017)
- Calixto and Liu (2017)
- Firat et al. (2016)
- Garmash and Monz (2016)
- Zoph and Knight (2016)
- Ha et al. (2016)
Multi-modal (speech, vision)
- Hitschler et al. (2016)
- Calixto et al. (2017)