This is how we do it: Answer Reranking for Open-domain How Questions with Paragraph Vectors and Minimal Feature Engineering

Proceedings of NAACL-HLT 2016, pages 1290–1295, San Diego, California, June 12-17, 2016. c⃝2016 Association for Computational Linguistics This is how we do it: Answer Reranking for Open-domain How Questions with Paragraph Vectors and Minimal Feature Engineering Dasha Bogdanova and Jennifer Foster ADAPT Centre School of Computing, Dublin City University Dublin, Ireland {dbogdanova,jfoster}@computing.dcu.ie Abstract We present a simple yet powerful approach to non-factoid answer reranking whereby question-answer pairs are represented by con- catenated distributed representation vectors and a multilayer perceptron is used to compute the score for an answer. Despite its simplicity, our approach achieves state-of-the-art perfor- mance on a public dataset of How questions, outperforming systems which employ sophis- ticated feature sets. We attribute this good per- formance to the use of paragraph instead of word vector representations and to the use of suitable data for training these representations. 1 Introduction In contrast to factoid question answering (QA), non- factoid QA is concerned with questions whose an- swer is not easily expressed as an entity or list of en- tities and can instead be quite complex – compare, for example, the factoid question Who is the secre- tary general of the UN? with the non-factoid manner question How is the secretary general of the UN cho- sen? A significant amount of research has been car- ried out on factoid QA, with non-factoid questions receiving less attention. This is changing, however, with the popularity of community-based question answering (CQA) sites such as Yahoo! Answers1, Quora2 and the StackExchange3 family of forums. The ability of users to vote for their favourite an- swer makes these sites a valuable source of training data for open-domain non-factoid QA systems. 1http://answers.yahoo.com 2http://quora.com 3http://stackexchange.com/ In this paper, we present a neural approach to open-domain non-factoid QA, focusing on the sub- task of answer reranking, i.e. given a list of can- didate answers to a question, order the answers according to their relevance to the question. We test our approach on the Yahoo! Answers dataset of manner or How questions introduced by Jansen et al. (2014), who describe answer reranking ex- periments on this dataset using a diverse range of features incorporating syntax, lexical semantics and discourse. In particular, they show how discourse in- formation (obtained either via a discourse parser or using shallow techniques based on discourse mark- ers) can complement distributed lexical semantic in- formation. Sharp et al. (2015) show how discourse structure can be used to generate artificial question- answer training pairs from documents, and test their approach on the same dataset. The best performance on this dataset – 33.01 P@1 and 53.96 MRR – is re- ported by Fried et al. (2015) who improve on the lexical semantic models of Jansen et al. (2014) by exploiting indirect associations between words us- ing higher-order models. In contrast, our approach is very simple and re- quires no feature engineering. Question-answer pairs are represented by concatenated distributed representation vectors and a multilayer perceptron is used to compute the score for an answer (the proba- bility of …

Content truncated to 500 words