Joint Speech Translation and Named Entity Recognition

Modern automatic translation systems aim at supporting the users by providing contextual knowledge. In this framework, a critical task is the output enrichment with information regarding the mentioned entities. This is currently achieved by processing the generated translations with named entity recognition (NER) tools and retrieving their description from knowledge bases. In light of the recent promising results shown by direct speech translation (ST) models and the known weaknesses of cascades (error propagation and additional latency), in this paper we propose multitask models that jointly perform ST and NER, and compare them with a cascade baseline. Experimental results on three language pairs (en-es/fr/it) show that our models significantly outperform the cascade on the NER task (by 0.4-1.0 F1), without degradation in terms of translation quality, and with the same computational efficiency of a plain direct ST model.