This paper investigates the problem of updating over time the statistical language model (LM) of an Italian broadcast news transcription system. Statistical adaptation methods are proposed which try to cope with the complex dynamics of news by exploiting newswire texts daily available on the Internet. In particular, contemporary news reports are used to extend the lexicon of the LM, to minimize the out-of-vocabulary (OOV) word rate, and to adapt the n-gram probabilities. Experiments performed on 19 news shows, spanning a period of one month, showed relative reductions of 58% in OOV word rate, 16% in perplexity, and 4% in word error rate (WER)
Broadcast news LM adaptation over time
Federico, Marcello;Bertoldi, Nicola
2004-01-01
Abstract
This paper investigates the problem of updating over time the statistical language model (LM) of an Italian broadcast news transcription system. Statistical adaptation methods are proposed which try to cope with the complex dynamics of news by exploiting newswire texts daily available on the Internet. In particular, contemporary news reports are used to extend the lexicon of the LM, to minimize the out-of-vocabulary (OOV) word rate, and to adapt the n-gram probabilities. Experiments performed on 19 news shows, spanning a period of one month, showed relative reductions of 58% in OOV word rate, 16% in perplexity, and 4% in word error rate (WER)I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
