In previous work we have proved that the Bleu algorithm (Papineni et al. 2001), originally devised for evaluating Machine Translation systems, can be applied to assessing short essays written by students. In this paper we present a comparative evaluation between this Bleu-inspired algorithm and a system based on Latent Semantic Analysis. In addition we propose an effective combination schema for them. Despite the simplicity of these shallow NLP methods, they achieve state-of-the-art correlations to the teachers` scores while keeping the language-independence and without requiring any domain specific knowledge.

Automatic Assessment of Students` free-text Answers underpinned by the Combination of a Bleu-inspired algorithm and LSA

Gliozzo, Alfio Massimiliano;Strapparava, Carlo;Magnini, Bernardo
2005

Abstract

In previous work we have proved that the Bleu algorithm (Papineni et al. 2001), originally devised for evaluating Machine Translation systems, can be applied to assessing short essays written by students. In this paper we present a comparative evaluation between this Bleu-inspired algorithm and a system based on Latent Semantic Analysis. In addition we propose an effective combination schema for them. Despite the simplicity of these shallow NLP methods, they achieve state-of-the-art correlations to the teachers` scores while keeping the language-independence and without requiring any domain specific knowledge.
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11582/3991
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo

Citazioni
  • ???jsp.display-item.citation.pmc??? ND
social impact