Fine-grained Linguistic Evaluation of Question Answering Systems
7973Authors
nan
Abstract
AbstractQuestion answering systems are complex systems using natural language processing. Some evaluation campaigns are organized to evaluate such systems in order to propose a classification of systems based on final results (number of correct answers). Nevertheless, teams need to evaluate more precisely the results obtained by their systems if they want to do a diagnostic evaluation. There are no tools or methods to do these evaluations systematically. We present REVISE, a tool for glass box evaluation based on diagnostic of question answering system results.
Keywords
Full Content
nan
Labeled Text
AbstractQuestion answering systems are complex systems using natural language processing. Some evaluation campaigns are organized to evaluate such systems in order to propose a classification of systems based on final results (number of correct answers). Nevertheless, teams need to evaluate more precisely the results obtained by their systems if they want to do a diagnostic evaluation. There are no tools or methods to do these evaluations systematically. We present REVISE, a tool for glass box evaluation based on diagnostic of question answering system results.
Publication Details
- Paper ID:
- 7973
- Year:
- 2010
- Publication:
- TIPSTER
- Publication Type:
- conf
- DOI:
- nan