This work presents the combination of Latent Semantic Analysis (LSA) with other Natural Language Processing techniques (stemming, removal of closed-class words and Word Sense Disambiguation) to improve the automatic assessment of students’ free-text answers. The combinational schema has been tested in the experimental framework provided by the free-text Computer Assisted Assessment (CAA) system called Atenea (Alfonseca and Pérez, 2004). This system is able to ask randomly or according to the students’ profile an open-ended question to the student and then, assign a score to it. The results prove that for all datasets, when the NLP techniques are combined with LSA, the Pearson correlation between the scores given by Atenea and the scores given by the teachers for the same dataset of questions improves. We believe that this is due to the complementarity between LSA, which works more at a shallow semantic level, and the rest of the NLP techniques used in Atenea, which are more focused on the lexical and syntactical levels.
About the effects of combining Latent Semantic Analysis with Natural Language Processing techniques for free-text assessment
Gliozzo, Alfio Massimiliano;Strapparava, Carlo;Magnini, Bernardo;
2005-01-01
Abstract
This work presents the combination of Latent Semantic Analysis (LSA) with other Natural Language Processing techniques (stemming, removal of closed-class words and Word Sense Disambiguation) to improve the automatic assessment of students’ free-text answers. The combinational schema has been tested in the experimental framework provided by the free-text Computer Assisted Assessment (CAA) system called Atenea (Alfonseca and Pérez, 2004). This system is able to ask randomly or according to the students’ profile an open-ended question to the student and then, assign a score to it. The results prove that for all datasets, when the NLP techniques are combined with LSA, the Pearson correlation between the scores given by Atenea and the scores given by the teachers for the same dataset of questions improves. We believe that this is due to the complementarity between LSA, which works more at a shallow semantic level, and the rest of the NLP techniques used in Atenea, which are more focused on the lexical and syntactical levels.I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
