Neural Machine Translation (NMT) has been shown to be more effective in translation tasks compared to phrase-based statistical machine translation. However, NMT systems are limited in translating low-resourced languages, due to the significant amount of parallel data that is required to learn useful mappings between languages. In this work, we show how so-called multilingual NMT can help to tackle the challenges associated with low-resourced language translation. The principle of multilingual NMT is to force the creation of hidden representations of words in a shared semantic space across multiple languages, thus enabling a positive parameter transfer across languages. In particular, we present multilingual translation experiments with three languages (English, Italian, Romanian) covering six translation directions, utilizing both recurrent neural networks and transformer (or self-attentive) neural networks. We then focus on the zero-shot translation problem, that is how to leverage multi-lingual data in order to learn translation directions uncovered by the data. Hence, we introduce our recently proposed iterative self-training method, that incrementally improves a multilingual NMT on a zero-shot direction by just relying on monolingual data. Our results on TED talks data show that multilingualNMT outperforms conventional bilingual NMT, that the transformer NMT outperforms re-current NMT, and that zero-shot NMT outperforms conventional pivoting methods and even matches the performance of a fully-trained bilingual system

Multilingual Neural Machine Translation for Low-Resource Languages

Surafel M. Lakew
;
Marcello Federico;Matteo Negri;Marco Turchi
2018

Abstract

Neural Machine Translation (NMT) has been shown to be more effective in translation tasks compared to phrase-based statistical machine translation. However, NMT systems are limited in translating low-resourced languages, due to the significant amount of parallel data that is required to learn useful mappings between languages. In this work, we show how so-called multilingual NMT can help to tackle the challenges associated with low-resourced language translation. The principle of multilingual NMT is to force the creation of hidden representations of words in a shared semantic space across multiple languages, thus enabling a positive parameter transfer across languages. In particular, we present multilingual translation experiments with three languages (English, Italian, Romanian) covering six translation directions, utilizing both recurrent neural networks and transformer (or self-attentive) neural networks. We then focus on the zero-shot translation problem, that is how to leverage multi-lingual data in order to learn translation directions uncovered by the data. Hence, we introduce our recently proposed iterative self-training method, that incrementally improves a multilingual NMT on a zero-shot direction by just relying on monolingual data. Our results on TED talks data show that multilingualNMT outperforms conventional bilingual NMT, that the transformer NMT outperforms re-current NMT, and that zero-shot NMT outperforms conventional pivoting methods and even matches the performance of a fully-trained bilingual system
File in questo prodotto:
File Dimensione Formato  
Multilingual_Neural_Machine_Translation_for_Low-Re.pdf

accesso aperto

Tipologia: Documento in Pre-print
Licenza: DRM non definito
Dimensione 310.74 kB
Formato Adobe PDF
310.74 kB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: http://hdl.handle.net/11582/319291
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
social impact