Incorporating Label Dependency for Answer Quality Tagging in Community Question Answering via CNN-LSTM-CRF

Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 1231–1241, Osaka, Japan, December 11-17 2016. Incorporating Label Dependency for Answer Quality Tagging in Community Question Answering via CNN-LSTM-CRF Yang Xiang, Xiaoqiang Zhou1, Qingcai Chen2, Zhihui Zheng, Buzhou Tang, Xiaolong Wang, and Yang Qin Intelligent Computation Research Center, Harbin Institute of Technology Shenzhen Graduate School, China {xiangyang.hitsz, xiaoqiang.jeseph,qingcai.chen, zhihui.zchina,tangbuzhou}@gmail.com wangxl@insun.hit.edu.cn, csyqin@hitsz.edu.cn Abstract In community question answering (cQA), the quality of answers are determined by the matching degree between question-answer pairs and the correlation among the answers. In this paper, we show that the dependency between the answer quality labels also plays a pivotal role. To validate the effectiveness of label dependency, we propose two neural network-based models, with dif- ferent combination modes of Convolutional Neural Networks, Long Short Term Memory and Conditional Random Fields. Extensive experiments are taken on the dataset released by the SemEval-2015 cQA shared task. The first model is a stacked ensemble of the networks. It achieves 58.96% on macro averaged F1, which improves the state-of-the-art neural network- based method by 2.82% and outperforms the Top-1 system in the shared task by 1.77%. The second is a simple attention-based model whose input is the connection of the question and its corresponding answers. It produces promising results with 58.29% on overall F1 and gains the best performance on the Good and Bad categories. 1 Introduction Community question answering (cQA) provide abundant human-to-human questions and answers, be- having as a good resource for building automatic question answering systems. For example, many users ask about “How to build your body like a model?” in Yahoo Answers3 and also there are many answerers who are willing to share their experiences. Answer quality tagging refers to automatically identifying whether an answer is good, so as to collect high quality question answer pairs. The task is challenging in that one should develop useful features that can effectively bridge the se- mantic gap between the question and answer (QA) pair. The matching degree on content is the primary factor that determines how good an answer is. It can be measured through a series of syntactic and semantic features such as overlapped linguistic elements (Berger et al., 2000; Agichtein et al., 2008; Punyakanok et al., 2004). For example, if the question and answer have several words shared, the answer is likely to be good. However, the content similarity performs well only when both the question and answer are well-structured and overlapped, which are rare to see in cQA sentences. Many previous studies discover specific features based on the characteristics of cQA systems. For instance, opinion leader or experts would probably contribute more good answers than bad, and the number of thumbs-up towards an answer may reflect how much the users accept it (Agichtein et al., 1 The first two authors have equal contributions. 2 Corresponding author 3 https://answers.yahoo.com/question/index?qid=20070918225025AA5Jz0G This work is licensed under a Creative Commons Attribution 4.0 International Licence. Licence details: http://creativecom- mons.org/licenses/by/4.0/ 1231 2008). The inefficiency of textual similarity …

Content truncated to 500 words