Spinning Straw into Gold: Using Free Text to Train Monolingual Alignment Models for Non-factoid Question Answering

Human Language Technologies: The 2015 Annual Conference of the North American Chapter of the ACL, pages 231–237, Denver, Colorado, May 31 – June 5, 2015. c⃝2015 Association for Computational Linguistics Spinning Straw into Gold: Using Free Text to Train Monolingual Alignment Models for Non-factoid Question Answering Rebecca Sharp1, Peter Jansen1, Mihai Surdeanu1, and Peter Clark2 1 University of Arizona, Tucson, AZ, USA 2 Allen Institute for Artificial Intelligence, Seattle, WA, USA {bsharp,pajansen,msurdeanu}@email.arizona.edu peterc@allenai.org Abstract Monolingual alignment models have been shown to boost the performance of question answering systems by ”bridging the lexical chasm” between questions and answers. The main limitation of these approaches is that they require semistructured training data in the form of question-answer pairs, which is diffi- cult to obtain in specialized domains or low- resource languages. We propose two inex- pensive methods for training alignment mod- els solely using free text, by generating ar- tificial question-answer pairs from discourse structures. Our approach is driven by two rep- resentations of discourse: a shallow sequen- tial representation, and a deep one based on Rhetorical Structure Theory. We evaluate the proposed model on two corpora from differ- ent genres and domains: one from Yahoo! Answers and one from the biology domain, and two types of non-factoid questions: man- ner and reason. We show that these align- ment models trained directly from discourse structures imposed on free text improve per- formance considerably over an information re- trieval baseline and a neural network language model trained on the same data. 1 Introduction Question Answering (QA) is a challenging task that draws upon many aspects of NLP. Unlike search or information retrieval, answers infrequently con- tain lexical overlap with the question (e.g. What should we eat for breakfast? – Zoe’s Diner has good pancakes), and require QA models to draw upon more complex methods to bridge this ”lexical chasm” (Berger et al., 2000). These methods range from robust shallow models based on lexical seman- tics, to deeper, explainably-correct, but much more brittle inference methods based on first order logic. Berger et al. (2000) proposed that this ”lexical chasm” might be partially bridged by repurposing statistical machine translation (SMT) models for QA. Instead of translating text from one language to another, these monolingual alignment models learn to translate from question to answer1, learning com- mon associations from question terms such as eat or breakfast to answer terms like kitchen, pancakes, or cereal. While monolingual alignment models have en- joyed a good deal of recent success in QA (see related work), they have expensive training data requirements, requiring a large set of aligned in- domain question-answer pairs for training. For low- resource languages or specialized domains like sci- ence or biology, often the only option is to enlist a domain expert to generate gold QA pairs – a process that is both expensive and time consuming. All of this means that only in rare cases are we accorded the luxury of having enough high-quality QA pairs to properly train an alignment model, and so these …

Content truncated to 500 words