Comparing Methodologies for Evaluating the Generator in a Speech-to-Speech Translation System

This paper compares two formal methods for evaluating the output of generation systems. In one case humans are used for hand coding and assessment, while in the other case they also perform the task of producing the reference text. It is argued that the methodology that tries to limit the human intervention gives less accurate results