Higher-order Lexical Semantic Models for Non-factoid Answer Reranking

Higher-order Lexical Semantic Models for Non-factoid Answer Reranking Daniel Fried1, Peter Jansen1, Gustave Hahn-Powell1, Mihai Surdeanu1, and Peter Clark2 1 University of Arizona, Tucson, AZ, USA 2 Allen Institute for Artificial Intelligence, Seattle, WA, USA {dfried,pajansen,hahnpowell,msurdeanu}@email.arizona.edu peterc@allenai.org Abstract Lexical semantic models provide robust performance for question answering, but, in general, can only capitalize on direct ev- idence seen during training. For example, monolingual alignment models acquire term alignment probabilities from semi- structured data such as question-answer pairs; neural network language models learn term embeddings from unstructured text. All this knowledge is then used to estimate the semantic similarity between question and answer candidates. We in- troduce a higher-order formalism that al- lows all these lexical semantic models to chain direct evidence to construct indirect associations between question and answer texts, by casting the task as the traversal of graphs that encode direct term associa- tions. Using a corpus of 10,000 questions from Yahoo! Answers, we experimentally demonstrate that higher-order methods are broadly applicable to alignment and lan- guage models, across both word and syn- tactic representations. We show that an important criterion for success is control- ling for the semantic drift that accumu- lates during graph traversal. All in all, the proposed higher-order approach improves five out of the six lexical semantic mod- els investigated, with relative gains of up to +13% over their first-order variants. 1 Introduction Open-domain question answering (QA), which finds short textual answers to natural language questions, is often viewed as the successor to key- word search (Etzioni, 2011) and one of the most difficult and widely applicable end-user applica- tions of natural language processing (NLP). From syntactic parsing, discourse processing, and lex- ical semantics, QA necessitates a level of func- tionality across a variety of topics that make it a natural, yet challenging, proving ground for many aspects of NLP. Here, we address a partic- ularly challenging QA subtask: open-domain non- factoid QA, where queries take the form of com- plex questions (e.g., manner or How questions), and answers range from single sentences to en- tire paragraphs. Because this task is so complex and large in scope, current state-of-the-art open- domain systems perform at only about 30% P@1, or answering roughly one out of three questions correctly (Jansen et al., 2014). In this paper we focus on answer ranking (AR), a key component of non-factoid QA that focuses on ordering candidate answers based on the like- lihood that they capture the information needed to answer a question. Unlike keyword search, Berger (2000) observed that lexical matching methods are generally insufficient for QA, where questions and answers often have little to no lexical overlap (as in the case of Where should we go for breakfast? and Zoe’s Diner has great pancakes). Previous work has shown that lexical semantics (LS) mod- els are well suited to bridging this “lexical chasm”, and at least two flavors of lexical semantics have been successfully applied to QA. The first treats QA as a monolingual alignment problem, learning associations between words …

Content truncated to 500 words