Named Entity Recognition in Question Answering of Speech Data

Named Entity Recognition in Question Answering of Speech Data Diego Moll´a Menno van Zaanen Steve Cassidy Division of ICS Department of Computing Macquarie University North Ryde Australia {diego,menno,cassidy}@ics.mq.edu.au Abstract Question answering on speech transcripts (QAst) is a pilot track of the CLEF com- petition. In this paper we present our con- tribution to QAst, which is centred on a study of Named Entity (NE) recognition on speech transcripts, and how it impacts on the accuracy of the final question answering system. We have ported AFNER, the NE recogniser of the AnswerFinder question- answering project, to the set of answer types expected in the QAst track. AFNER uses a combination of regular expressions, lists of names (gazetteers) and machine learning to find NeWS in the data. The machine learn- ing component was trained on a develop- ment set of the AMI corpus. In the process we identified various problems with scala- bility of the system and the existence of er- rors of the extracted annotation, which lead to relatively poor performance in general. Performance was yet comparable with state of the art, and the system was second (out of three participants) in one of the QAst sub- tasks. 1 Introduction AnswerFinder is a question answering system that focuses on shallow semantic representations of questions and text (Moll´a and van Zaanen, 2006; van Zaanen et al., 2007). The underlying idea of AnswerFinder is that the use of semantic represen- tations reduces the impact of paraphrases (different wordings of the same information). The system uses symbolic algorithms to find exact answers to ques- tions in large document collections. The design and implementation of the An- swerFinder system has been driven by requirements that the system should be easy to configure, extend, and, therefore, port to new domains. To measure the success of the implementation of AnswerFinder in these respects, we decided to participate in the CLEF 2007 pilot task of question answering on speech transcripts (QAst). The task in this compe- tition is different from that for which AnswerFinder was originally designed and provides a good test of portability to new domains. The current CLEF pilot track QAst presents an interesting and challenging new application of ques- tion answering. The objective in QAst is to answer questions based on transcripts of meetings and lec- tures. Both automatic and manual transcripts are provided; the automatic transcripts being the result of applying a speech recogniser to the audio record- ings. The data for the task is taken from corpora collected by the AMI (Augmented Multiparty Inter- action) project (Carletta et al., 2005) and from the CHIL (Computers in the Human Interaction Loop) project (Waibel et al., 2004). While both corpora are extensively annotated, only speaker turn annotation is provided in the input data for this task. In our contribution we focus on adapting AFNER, our Named Entity Recogniser (NER), for speech transcripts and its application for Question Answer- ing. Named Entity (NE) recognition is the task of finding instances of specific types of entities in …

Content truncated to 500 words