Learning from errors is a crucial aspect of improving expertise. Based on this no- tion, we discuss a robust statistical frame- work for analysing the impact of different error types on machine translation (MT) output quality. Our approach is based on linear mixed-effects models, which allow the analysis of error-annotated MT out- put taking into account the variability in- herent to the specific experimental setting from which the empirical observations are drawn. Our experiments are carried out on different language pairs involving Chi- nese, Arabic and Russian as target lan- guages. Interesting findings are reported, concerning the impact of different error types both at the level of human perception of quality and with respect to performance results measured with automatic metrics.
Assessing the Impact of Translation Errors on Machine Translation Quality with Mixed-effects Models.
Federico, Marcello;Negri, Matteo;Bentivogli, Luisa;Turchi, Marco
2014-01-01
Abstract
Learning from errors is a crucial aspect of improving expertise. Based on this no- tion, we discuss a robust statistical frame- work for analysing the impact of different error types on machine translation (MT) output quality. Our approach is based on linear mixed-effects models, which allow the analysis of error-annotated MT out- put taking into account the variability in- herent to the specific experimental setting from which the empirical observations are drawn. Our experiments are carried out on different language pairs involving Chi- nese, Arabic and Russian as target lan- guages. Interesting findings are reported, concerning the impact of different error types both at the level of human perception of quality and with respect to performance results measured with automatic metrics.I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
