ECNU at SemEval-2016 Task 3: Exploring Traditional Method and Deep Learning Method for Question Retrieval and Answer Ranking in Community Question Answering

Proceedings of SemEval-2016, pages 872–878, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics ECNU at SemEval-2016 Task 3: Exploring Traditional Method and Deep Learning Method for Question Retrieval and Answer Ranking in Community Question Answering Guoshun Wu1, Man Lan1,2∗ 1Department of Computer Science and Technology, East China Normal University, Shanghai, P.R.China 2Shanghai Key Laboratory of Multidimensional Information Processing 51141201064@ecnu.cn, mlan@cs.ecnu.edu.cn∗ Abstract This paper describes the system we submit- ted to the task 3 (Community Question An- swering) in SemEval 2016, which contains three subtasks, i.e., Question-Comment Simi- larity (subtask A), Question-Question Similar- ity (subtask B), and Question-External Com- ment Similarity (subtask C). For subtask A, we employed three different methods to rank question-comment pair, i.e., supervised model using traditional features, Convolutional Neu- ral Network and Long-Short Term Memory Network. For subtask B, we proposed two novel methods to improve semantic similarity estimation between question-question pair by integrating the rank information of question- comment pair. For subtask C, we implement- ed a two-step strategy to select out the simi- lar questions and filter the unrelated comments with respect to the original question. 1 Introduction The purpose of Community Question Answering task in SemEval 2016 (Nakov et al., 2016) is to provide a platform for finding good answers to new questions in a community-created discussion forum, where the main task (subtask C) is defined as fol- lows: given a new question and a large collection of question-comment threads created by a user com- munity, participants are required to rank the com- ments that are most useful for answering the new question. Obviously, this main task consists of two optional subtasks, i.e., Question-Comment Similari- ty (subtask A, also known as answer ranking), which is to re-rank comments/answers according to their relevance with respect to the question, and Question- Question Similarity (i.e., subtask B, also known as question retrieval), which is to retrieve the similar questions according to their semantic similarity with respect to the original question. To address subtask A, we explored a traditional machine learning method which uses multiple types of features, e.g., Word Match Features, Translation- based Features, and Lexical Semantic Similarity Features. Additionally, for subtask A, we also built a Convolutional Neural Network (CNN) model and a bidirectional Long Short-Term Memory (BLST- M) model to learn joint representation for question- comment (Q-C) pair. For subtask B, besides IR method and traditional machine learning method, we also proposed two novel methods to improve seman- tic similarity estimation between question-question (Q-Q) pairs by integrating the rank information of Q-C pairs. Since subtask C can be regarded as a joint work of the two above-mentioned subtasks, we implemented a two-step strategy to first select out similar questions and then to filter out the unrelated comments with respect to the original question. The rest of this paper is organized as follows. Sec- tion 2 describes our system. Section 3 describes ex- perimental setting. Section 4 and 5 report results on training and test sets. Finally, Section 6 concludes this work. 2 …

Content truncated to 500 words