Learning Adaptable Patterns for Passage Reranking

Proceedings of the Seventeenth Conference on Computational Natural Language Learning, pages 75–83, Sofia, Bulgaria, August 8-9 2013. c⃝2013 Association for Computational Linguistics Learning Adaptable Patterns for Passage Reranking Aliaksei Severyn(1) and Massimo Nicosia(1) and Alessandro Moschitti1,2 (1)DISI, University of Trento, 38123 Povo (TN), Italy {severyn,m.nicosia,moschitti}@disi.unitn.it (2)QCRI, Qatar Foundation, 5825 Doha, Qatar amoschitti@qf.org.qa Abstract This paper proposes passage reranking models that (i) do not require manual fea- ture engineering and (ii) greatly preserve accuracy, when changing application do- main. Their main characteristic is the use of relational semantic structures rep- resenting questions and their answer pas- sages. The relations are established us- ing information from automatic classifiers, i.e., question category (QC) and focus classifiers (FC) and Named Entity Recog- nizers (NER). This way (i) effective struc- tural relational patterns can be automati- cally learned with kernel machines; and (ii) structures are more invariant w.r.t. dif- ferent domains, thus fostering adaptability. 1 Introduction A critical issue for implementing Question An- swering (QA) systems is the need of designing answer search and extraction modules specific to the target application domain. These modules en- code handcrafted rules based on syntactic patterns that detect the relations between a question and its candidate answers in text fragments. Such rules are triggered when patterns in the question and the passage are found. For example, given a ques- tion1: What is Mark Twain’s real name? and a relevant passage, e.g., retrieved by a search engine: Samuel Langhorne Clemens, better known as Mark Twain. the QA engineers typically apply a syntactic parser to obtain the parse trees of the above two sen- tences, from which, they extract rules like: 1We use this question/answer pair from TREC QA as a running example in the rest of the paper. if the pattern “What is NP2’s ADJ name” is in the question and the pat- tern “NP1 better known as NP2” is in the answer passage then associate the passage with a high score2. Machine learning has made easier the task of QA engineering by enabling the automatic learn- ing of answer extraction modules. However, new features and training data have to be typically de- veloped when porting a QA system from a domain to another. This is even more critical considering that effective features tend to be as much complex and similar as traditional handcrafted rules. To reduce the burden of manual feature engi- neering for QA, we proposed structural models based on kernel methods, (Moschitti et al., 2007; Moschitti and Quarteroni, 2008; Moschitti, 2008) with passages limited to one sentence. Their main idea is to: (i) generate question and passage pairs, where the text passages are retrieved by a search engine; (ii) assuming those containing the correct answer as positive instance pairs and all the oth- ers as negative ones; (iii) represent such pairs with two syntactic trees; and (ii) learn to rank answer passages by means of structural kernels applied to two trees. This enables the automatic engineering of structural/lexical semantic patterns. More recently, we showed that such models …

Content truncated to 500 words