Sequential Attention with Keyword Mask Model for Community-based Question Answering

Proceedings of NAACL-HLT 2019, pages 2201–2211 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 2201 Sequential Attention with Keyword Mask Model for Community-based Question Answering Jianxin Yang1,2, Wenge Rong1,2, Libin Shi3, Zhang Xiong1,2 1State Key Laboratory of Software Development Environment, Beihang University, China 2School of Computer Science and Engineering, Beihang University, China 3Microsoft, China {yjx17,w.rong,xiongz}@buaa.edu.cn, libshi@microsoft.com Abstract In Community-based Question Answering system(CQA), Answer Selection(AS) is a crit- ical task, which focuses on finding a suit- able answer within a list of candidate answers. For neural network models, the key issue is how to model the representations of QA text pairs and calculate the interactions between them. We propose a Sequential Attention with Keyword Mask model(SAKM) for CQA to imitate human reading behavior. Question and answer text regard each other as context within keyword-mask attention when encod- ing the representations, and repeat multiple times(hops) in a sequential style. So the QA pairs capture features and information from both question text and answer text, interact- ing and improving vector representations it- eratively through hops. The flexibility of the model allows to extract meaningful keywords from the sentences and enhance diverse mutual information. We perform on answer selection tasks and multi-level answer ranking tasks. Experiment results demonstrate the superiority of our proposed model on community-based QA datasets. 1 Introduction Answering selection(AS) is one of the most fun- damental challenges in community-based question answering(CQA) services. Given a question and a list of candidate answers, its aim is to choose the most matching one to the question. During this process of matching questions and answer candi- dates, how to encode the question and answer(QA) into meaningful and semantic representations im- pacts on the results directly. Earlier conventional statistic methods are nor- mally based on feature engineering and resource toolkits. Though these methods are easy in im- plementation, they require extra efforts and hand- crafted features(Heilman and Smith, 2010; Ty- moshenko and Moschitti, 2015). Recently, with the development of neural network, deep learn- ing based models attract much attention in vari- ous tasks(Krizhevsky et al., 2012; Sutskever et al., 2014). In question answering field, the convo- lutional neural networks(CNNs)(Yu et al., 2014; Hu et al., 2014; Severyn and Moschitti, 2015) and recurrent neural networks(RNNs)(Wang and Ny- berg, 2015; Feng et al., 2015) are widely employed to convert the question and answer text into vec- tors and define a feed-forward multi-layer percep- tron to compute the interactions between them. These models construct sentences in an end-to- end fashion with less manual involvement. To capture fine-grained features, on the one hand, some works are concerned with matching QA pairs relationship in a more complex and diverse way, e.g., CNTN(Qiu and Huang, 2015) and MV- LSTM(Wan et al., 2016). On the other hand, latent representation models aim to jointly learn lexical and semantic information from QA sentences and influence the vector generation directly, e.g., atten- tion mechanism(Bahdanau et al., 2015). Attention mechanism learns attention weights of each words pairs between QA sentences. Af- terwards …

Content truncated to 500 words