In this paper we present a method to evaluate Question Answering systems in restricted domain applications, which present peculiar features with respect to open domain Question Answering. While the answers provided by the system are still evaluated manually as in QA evaluation based on Information Retrieval, we propose to assess the overall quality of answers rather than concentrating merely on answers’ correctness. We propose to evaluate each answer from different perspectives, i.e. not only from the point of view of the exactness of the answer, but of the data provided as well, while at the same time taking into consideration aspects connected to comprehensibility and answer presentation.

Assessing Answer Quality in Task-oriented Question Answering

Speranza, Manuela;Gretter, Roberto
2009-01-01

Abstract

In this paper we present a method to evaluate Question Answering systems in restricted domain applications, which present peculiar features with respect to open domain Question Answering. While the answers provided by the system are still evaluated manually as in QA evaluation based on Information Retrieval, we propose to assess the overall quality of answers rather than concentrating merely on answers’ correctness. We propose to evaluate each answer from different perspectives, i.e. not only from the point of view of the exactness of the answer, but of the data provided as well, while at the same time taking into consideration aspects connected to comprehensibility and answer presentation.
2009
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11582/5212
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo

Citazioni
  • ???jsp.display-item.citation.pmc??? ND
social impact