We present the results from the fourth round of the WMT shared task on MTAutomatic Post-Editing. The task consists in automatically correcting the output of a “black-box” machine translation system by learning from human corrections. Keeping the same general evaluation setting of the three previous rounds, this year we focused on one language pair (English-German) and on domain-specific data (Information Technology), with MT outputs produced by two different paradigms: phrase-based (PBSMT) and neural (NMT). Five teams submitted respectively 11 runs for the PBSMT subtask and 10 runs for the NMT sub-task. In the former subtask, characterized by original translations of lower quality, top results achieved impressive improvements, up to -6.24 TER and +9.53 BLEU points over the baseline “do-nothing” system. The NMT subtask proved to be more challenging due to the higher quality of the original translations and the availability of less training data. In this case, top results show smaller improvements up to-0.38 TER and +0.8 BLEU points.

Findings of the WMT 2018 Shared Task on Automatic Post-Editing

Rajen Chatterjee;Matteo Negri;Marco Turchi
2018-01-01

Abstract

We present the results from the fourth round of the WMT shared task on MTAutomatic Post-Editing. The task consists in automatically correcting the output of a “black-box” machine translation system by learning from human corrections. Keeping the same general evaluation setting of the three previous rounds, this year we focused on one language pair (English-German) and on domain-specific data (Information Technology), with MT outputs produced by two different paradigms: phrase-based (PBSMT) and neural (NMT). Five teams submitted respectively 11 runs for the PBSMT subtask and 10 runs for the NMT sub-task. In the former subtask, characterized by original translations of lower quality, top results achieved impressive improvements, up to -6.24 TER and +9.53 BLEU points over the baseline “do-nothing” system. The NMT subtask proved to be more challenging due to the higher quality of the original translations and the availability of less training data. In this case, top results show smaller improvements up to-0.38 TER and +0.8 BLEU points.
2018
978-1-948087-81-0
File in questo prodotto:
File Dimensione Formato  
WMT080.pdf

accesso aperto

Tipologia: Documento in Post-print
Licenza: Creative commons
Dimensione 378.78 kB
Formato Adobe PDF
378.78 kB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11582/316423
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
social impact