Context-Aware Interaction Network for Question Matching

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3846–3853 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 3846 Context-Aware Interaction Network for Question Matching Zhe Hu1, Zuohui Fu2, Yu Yin3, and Gerard de Melo4 1Baidu Inc., China, 2Rutgers University, USA 3Northeastern University, USA, 4HPI/University of Potsdam, Germany 1huzhe01@baidu.com, 2zuohui.fu@rutgers.edu 3yin.yu1@northeastern.edu, 4gdm@demelo.org Abstract Impressive milestones have been achieved in text matching by adopting a cross-attention mechanism to capture pertinent semantic con- nections between two sentence representations. However, regular cross-attention focuses on word-level links between the two input se- quences, neglecting the importance of contex- tual information. We propose a context-aware interaction network (COIN) to properly align two sequences and infer their semantic rela- tionship. Specifically, each interaction block includes (1) a context-aware cross-attention mechanism to effectively integrate contextual information when aligning two sequences, and (2) a gate fusion layer to flexibly interpolate aligned representations. We apply multiple stacked interaction blocks to produce align- ments at different levels and gradually refine the attention results. Experiments on two ques- tion matching datasets and detailed analyses demonstrate the effectiveness of our model. 1 Introduction Semantic text matching is among the most funda- mental tasks in natural language processing. Given two sentences, the goal is to predict their semantic relationship. In this work, we focus in particular on question matching (QM) benchmarks. Recently, the availability of large-scale anno- tated datasets has led to a proliferation of deep neural architectures for text matching (Williams et al., 2018; Chen et al., 2017; Wang et al., 2017). Most existing neural models fall into two cate- gories, namely the sentence encoding and the sen- tence interaction approaches (Lan and Xu, 2018). The former encodes sentences as fixed-length vec- tor representations, which are then consulted to make the final prediction. The latter considers in- teractions between two sequences to identify their semantic connections, which tends to yield better results. !"# !"$ !"% !"& !'# !'$ !'% !'& !"# !"$ !"% !"& !'# !'$ !'% !'& (' (" )** )**+ (a) Original Attention (b) Context-aware Attention Figure 1: The original attention mechanism (left) and the proposed context-aware attention (right). w∗ represents the two sequences (more generally, they can be regarded as query and key). C∗ denotes the contextual features. Attention mechanisms are widely adopted for the sentence-interaction approaches, relying on a word-by-word attention matrix to obtain align- ment information between two sequences. This has proven fruitful in modeling sentence pair rela- tionships (Parikh et al., 2016; Rocktäschel et al., 2015; Wang and Jiang, 2016). Nonetheless, when computing the cross-sentence attention, existing models mostly focus on word-level local matching and fail to fully account for the overall semantics: each value of the attention matrix is based on just two individual tokens from the sequences without full consideration of the context. As shown in Fig- ure 1, in the original attention mechanism, each to- ken individually attends to the other tokens without accounting for important contextual information. However, accurate matching may require a deeper understanding of the two …

Content truncated to 500 words