Simple and Effective Text Matching with Richer Alignment Features

Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4699–4709 Florence, Italy, July 28 - August 2, 2019. c⃝2019 Association for Computational Linguistics 4699 Simple and Effective Text Matching with Richer Alignment Features Runqi Yang1, Jianhai Zhang2, Xing Gao2, Feng Ji2, Haiqing Chen2 1Department of Computer Science and Technology, Nanjing University, China runqiyang@gmail.com 2Alibaba Group, Hangzhou, China {tanfan.zjh,gaoxing.gx,zhongxiu.jf, haiqing.chenhq}@alibaba-inc.com Abstract In this paper, we present a fast and strong neu- ral approach for general purpose text matching applications. We explore what is sufficient to build a fast and well-performed text matching model and propose to keep three key features available for inter-sequence alignment: origi- nal point-wise features, previous aligned fea- tures, and contextual features while simplify- ing all the remaining components. We conduct experiments on four well-studied benchmark datasets across tasks of natural language in- ference, paraphrase identification and answer selection. The performance of our model is on par with the state-of-the-art on all datasets with much fewer parameters and the inference speed is at least 6 times faster compared with similarly performed ones. 1 Introduction Text matching is a core research area in natural language processing with a long history. In text matching tasks, a model takes two text sequences as input and predicts a category or a scala value in- dicating their relationship. A wide range of tasks, including natural language inference (also known as recognizing textual entailment) (Bowman et al., 2015; Khot et al., 2018), paraphrase identification (Wang et al., 2017), answer selection (Yang et al., 2015), and so on, can be seen as specific forms of text matching problems. Research on general purpose text matching algorithm is beneficial to a large number of relevant applications. Deep neural networks are the most popular choices for text matching nowadays. Semantic alignment and comparison of two text sequences are the keys in neural text matching. Many pre- vious deep neural networks contain a single inter- sequence alignment layer. To make full use of this only alignment process, the model has to take rich external syntactic features or hand-designed align- ment features as additional inputs of the alignment layer (Chen et al., 2017; Gong et al., 2018), adopt a complicated alignment mechanism (Wang et al., 2017; Tan et al., 2018), or build a vast amount of post-processing layers to analyze the alignment re- sult (Tay et al., 2018b; Gong et al., 2018). More powerful models can be built with mul- tiple inter-sequence alignment layers. Instead of making a prediction based on the comparison re- sult of a single alignment process, a stacked model with multiple alignment layers maintains its in- termediate states and gradually refines its predic- tions. However, suffering from inefficient propa- gation of lower-level features and vanishing gradi- ents, these deeper architectures are harder to train. Recent works have come up with ways of connect- ing stacked building blocks including dense con- nection (Tay et al., 2018a; Kim et al., 2018) and recurrent neural networks (Liu et al., 2018), which strengthen the propagation of …

Content truncated to 500 words