Bridging the Gap between Relevance Matching and Semantic Matching for Short Text Similarity Modeling

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 5370–5381, Hong Kong, China, November 3–7, 2019. c⃝2019 Association for Computational Linguistics 5370 Bridging the Gap Between Relevance Matching and Semantic Matching for Short Text Similarity Modeling Jinfeng Rao,1 Linqing Liu,2 Yi Tay,3 Wei Yang,2 Peng Shi,2 and Jimmy Lin2 1 Facebook Assistant 2 David R. Cheriton School of Computer Science, University of Waterloo 3 Nanyang Technological University raojinfeng@fb.com Abstract A core problem of information retrieval (IR) is relevance matching, which is to rank doc- uments by relevance to a user’s query. On the other hand, many NLP problems, such as question answering and paraphrase iden- tification, can be considered variants of se- mantic matching, which is to measure the se- mantic distance between two pieces of short texts. While at a high level both relevance and semantic matching require modeling tex- tual similarity, many existing techniques for one cannot be easily adapted to the other. To bridge this gap, we propose a novel model, HCAN (Hybrid Co-Attention Network), that comprises (1) a hybrid encoder module that includes ConvNet-based and LSTM-based en- coders, (2) a relevance matching module that measures soft term matches with importance weighting at multiple granularities, and (3) a semantic matching module with co-attention mechanisms that capture context-aware se- mantic relatedness. Evaluations on multiple IR and NLP benchmarks demonstrate state-of- the-art effectiveness compared to approaches that do not exploit pretraining on external data. Extensive ablation studies suggest that rele- vance and semantic matching signals are com- plementary across many problem settings, re- gardless of the choice of underlying encoders. 1 Introduction Neural networks have achieved great success in many NLP tasks, such as question answering (Rao et al., 2016; Chen et al., 2017a), paraphrase de- tection (Wang et al., 2017), and textual semantic similarity modeling (He and Lin, 2016). Many of these tasks can be treated as variants of a seman- tic matching (SM) problem, where two pieces of texts are jointly modeled through distributed rep- resentations for similarity learning. Various neural network architectures, e.g., Siamese networks (He et al., 2016) and attention (Seo et al., 2017; Tay et al., 2019b), have been proposed to model se- mantic similarity using diverse techniques. A core problem of information retrieval (IR) is relevance matching (RM), where the goal is to rank documents by relevance to a user’s query. Though at a high level semantic and relevance matching both require modeling similarities in pairs of texts, there are fundamental differences. Semantic matching emphasizes “meaning” corre- spondences by exploiting lexical information (e.g., words, phrases, entities) and compositional struc- tures (e.g., dependency trees), while relevance matching focuses on keyword matching. It has been observed that existing approaches for tex- tual similarity modeling in NLP can produce poor results for IR tasks (Guo et al., 2016), and vice versa (Htut et al., 2018). Specifically, Guo et al. (2016) point out three distinguishing characteristics of relevance match- ing: exact match signals, …

Content truncated to 500 words