Discourse Complements Lexical Semantics for Non-factoid Answer Reranking

Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pages 977–986, Baltimore, Maryland, USA, June 23-25 2014. c⃝2014 Association for Computational Linguistics Discourse Complements Lexical Semantics for Non-factoid Answer Reranking Peter Jansen and Mihai Surdeanu University of Arizona Tucson, AZ, USA {pajansen,msurdeanu} @email.arizona.edu Peter Clark Allen Institute for Artificial Intelligence Seattle, WA, USA peterc@allenai.org Abstract We propose a robust answer reranking model for non-factoid questions that inte- grates lexical semantics with discourse in- formation, driven by two representations of discourse: a shallow representation cen- tered around discourse markers, and a deep one based on Rhetorical Structure Theory. We evaluate the proposed model on two corpora from different genres and domains: one from Yahoo! Answers and one from the biology domain, and two types of non-factoid questions: manner and reason. We experimentally demon- strate that the discourse structure of non- factoid answers provides information that is complementary to lexical semantic sim- ilarity between question and answer, im- proving performance up to 24% (relative) over a state-of-the-art model that exploits lexical semantic similarity alone. We fur- ther demonstrate excellent domain transfer of discourse information, suggesting these discourse features have general utility to non-factoid question answering. 1 Introduction Driven by several international evaluations and workshops such as the Text REtrieval Conference (TREC)1 and the Cross Language Evaluation Fo- rum (CLEF),2 the task of question answering (QA) has received considerable attention. However, most of this effort has focused on factoid questions rather than more complex non-factoid (NF) ques- tions, such as manner, reason, or causation ques- tions. Moreover, the vast majority of QA mod- els explore only local linguistic structures, such as syntactic dependencies or semantic role frames, 1http://trec.nist.gov 2http://www.clef-initiative.eu which are generally restricted to individual sen- tences. This is problematic for NF QA, where questions are answered not by atomic facts, but by larger cross-sentence conceptual structures that convey the desired answers. Thus, to answer NF questions, one needs a model of what these answer structures look like. Driven by this observation, our main hypothe- sis is that the discourse structure of NF answers provides complementary information to state-of- the-art QA models that measure the similarity (ei- ther lexical and/or semantic) between question and answer. We propose a novel answer reranking (AR) model that combines lexical semantics (LS) with discourse information, driven by two rep- resentations of discourse: a shallow representa- tion centered around discourse markers and sur- face text information, and a deep one based on the Rhetorical Structure Theory (RST) discourse framework (Mann and Thompson, 1988). To the best of our knowledge, this work is the first to systematically explore within- and cross-sentence structured discourse features for NF AR. The con- tributions of this work are: 1. We demonstrate that modeling discourse is greatly beneficial for NF AR for two types of NF questions, manner (“how”) and rea- son (“why”), across two large datasets from different genres and domains – one from the community question-answering (CQA) site of Yahoo! Answers3, and one from a biology textbook. Our results …

Content truncated to 500 words