KeLP at SemEval-2017 Task 3: Learning Pairwise Patterns in Community Question Answering

Proceedings of the 11th International Workshop on Semantic Evaluations (SemEval-2017), pages 326–333, Vancouver, Canada, August 3 - 4, 2017. c⃝2017 Association for Computational Linguistics KeLP at SemEval-2017 Task 3: Learning Pairwise Patterns in Community Question Answering Simone Filice1, Giovanni Da San Martino2 and Alessandro Moschitti2 1 DICII, University of Roma, Tor Vergata 2 ALT, Qatar Computing Research Institute, HBKU filice.simone@gmail.com {gmartino,amoschitti}@hbku.edu.qa Abstract This paper describes the KeLP system par- ticipating in the SemEval-2017 commu- nity Question Answering (cQA) task. The system is a refinement of the kernel-based sentence pair modeling we proposed for the previous year challenge. It is imple- mented within the Kernel-based Learning Platform called KeLP, from which we in- herit the team’s name. Our primary sub- mission ranked first in subtask A, and third in subtasks B and C, being the only sys- tems appearing in the top-3 ranking for all the English subtasks. This shows that the proposed framework, which has minor variations among the three subtasks, is ex- tremely flexible and effective in tackling learning tasks defined on sentence pairs. 1 Introduction This paper describes the KeLP system participat- ing in the SemEval-2017 cQA challenge (Nakov et al., 2017). The task setting for the English part is the same as the previous edition (Nakov et al., 2016): the corpus is extracted from Qatar Living1, a web forum where people pose questions about multiple aspects of their daily life in Qatar, and three subtasks are defined: Subtask A: Given a question q and its first 10 comments c1, . . . , c10 in its question thread, re- rank these 10 comments according to their rel- evance with respect to the question, i.e., the good comments have to be ranked above poten- tial or bad comments. Subtask B: Given a new question o and the set of the first 10 related questions q1, . . . , q10 (re- trieved by a search engine), re-rank the related questions according to their similarity with respect 1http://www.qatarliving.com/forum to o, i.e., the perfect match and relevant questions should be ranked above the irrelevant ones. Subtask C: Given a new question o, and the set of the first 10 related questions, q1, . . . , q10, (re- trieved by a search engine), each one associated with its first 10 comments, cq 1, . . . , cq 10, appearing in its thread, re-rank the 100 comments according to their relevance with respect to o, i.e., the good comments are to be ranked above potential or bad comments. We participated to the previous year edition, where our system (Filice et al., 2016) achieved very good results, i.e., first in subtask A, third in B and second in C. For the new year challenge, we therefore decided to reuse the same system ap- plied to a new method for selecting tree structures, (Barr´on-Cede˜no et al., 2016; Romeo et al., 2016) summarized in Sec. 3. We modeled the three subtasks as binary clas- sification problems: kernel-based classifiers are trained and the classification …

Content truncated to 500 words