IIT-UHH at SemEval-2017 Task 3: Exploring Multiple Features for Community Question Answering and Implicit Dialogue Identification

Proceedings of the 11th International Workshop on Semantic Evaluations (SemEval-2017), pages 90–97, Vancouver, Canada, August 3 - 4, 2017. c⃝2017 Association for Computational Linguistics IIT-UHH at SemEval-2017 Task 3: Exploring Multiple Features for Community Question Answering and Implicit Dialogue Identification Titas Nandi1, Chris Biemann2, Seid Muhie Yimam2, Deepak Gupta1, Sarah Kohail2, Asif Ekbal1 and Pushpak Bhattacharyya1 1Indian Institute of Technology Patna 2Universit¨at Hamburg Germany {titas.ee13,deepak.pcs16,asif,pb}@iitp.ac.in {biemann,yimam,kohail}@informatik.uni-hamburg.de Abstract In this paper we present the system for An- swer Selection and Ranking in Commu- nity Question Answering, which we build as part of our participation in SemEval- 2017 Task 3. We develop a Support Vector Machine (SVM) based system that makes use of textual, domain-specific, word- embedding and topic-modeling features. In addition, we propose a novel method for dialogue chain identification in com- ment threads. Our primary submission won subtask C, outperforming other sys- tems in all the primary evaluation met- rics. We performed well in other English subtasks, ranking third in subtask A and eighth in subtask B. We also developed open source toolkits for all the three En- glish subtasks by the name cQARank1. 1 Introduction This paper presents the system built for partic- ipation in the SemEval-2017 Shared Task 3 on Community Question Answering (CQA). The task aims to classify and rank a candidate text c in relevance to a target text t. Based on the na- ture of the candidate and target texts, the main task is subdivided into three subtasks in which the teams are expected to solve the problem of Question-Comment similarity, Question-Question similarity and Question-External Comment simi- larity (Nakov et al., 2017). In this work, we propose a rich feature-based sys- tem for solving these problems. We create an ar- chitecture which integrates textual, semantic and domain-specific features to achieve good results in the proposed task. Due to the extremely noisy na- ture of the social forum data, we also develop a 1https://github.com/TitasNandi/cQARank customized preprocessing pipeline, rather than us- ing the standard tools. We use Support Vector Ma- chine (SVM) (Cortes and Vapnik, 1995) for clas- sification, and its confidence score for ranking. We initially define a generic set of features to de- velop a robust system for all three subtasks, then include additional features based on the nature of the subtasks. To adapt the system to subtasks B and C, we include features extracted from the scores of the other subtasks, propagating mean- ingful information essential in an incremental set- ting. We propose a novel method for identifica- tion of dialogue groups in the comment thread by constructing a user interaction graph and also in- corporate features from this graph in our system. Our algorithm outputs mutually disjoint groups of users who are involved in conversation with each other in the comment thread. The rest of the paper is organized as follows: Sec- tion 2 describes the related work. Sections 3, 4, and 5 elucidate the system architecture, features used and algorithms developed. Section 6 pro- vides experimentation details and reports the of- ficial …

Content truncated to 500 words