Task-Specific Attentive Pooling of Phrase Alignments Contributes to Sentence Matching

Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 699–709, Valencia, Spain, April 3-7, 2017. c⃝2017 Association for Computational Linguistics Task-Specific Attentive Pooling of Phrase Alignments Contributes to Sentence Matching Wenpeng Yin, Hinrich Sch¨utze The Center for Information and Language Processing LMU Munich, Germany wenpeng@cis.lmu.de Abstract This work studies comparatively two typ- ical sentence matching tasks: textual en- tailment (TE) and answer selection (AS), observing that weaker phrase alignments are more critical in TE, while stronger phrase alignments deserve more attention in AS. The key to reach this observation lies in phrase detection, phrase represen- tation, phrase alignment, and more im- portantly how to connect those aligned phrases of different matching degrees with the final classifier. Prior work (i) has limitations in phrase generation and representation, or (ii) con- ducts alignment at word and phrase lev- els by handcrafted features or (iii) utilizes a single framework of alignment without considering the characteristics of specific tasks, which limits the framework’s effec- tiveness across tasks. We propose an architecture based on Gated Recurrent Unit that supports (i) rep- resentation learning of phrases of arbi- trary granularity and (ii) task-specific at- tentive pooling of phrase alignments be- tween two sentences. Experimental results on TE and AS match our observation and show the effectiveness of our approach. 1 Introduction How to model a pair of sentences is a critical is- sue in many NLP tasks, including textual entail- ment (Marelli et al., 2014a; Bowman et al., 2015a; Yin et al., 2016a) and answer selection (Yu et al., 2014; Yang et al., 2015; Santos et al., 2016). A key challenge common to these tasks is the lack of explicit alignment annotation between the sen- tences of the pair. Thus, inferring and assessing the semantic relations between words and phrases in the two sentences is a core issue. Figure 1: Alignment examples in TE (top) and AS (bottom). Green color: identical (subset) align- ment; blue color: relatedness alignment; red color: unrelated alignment. Q: the first sentence in TE or the question in AS; C+, C−: the correct or incor- rect counterpart in the sentence pair (Q, C). Figure 1 shows examples of human annotated phrase alignments. In the TE example, we try to figure out Q entails C+ (positive) or C− (nega- tive). As human beings, we discover the relation- ship of two sentences by studying the alignments between linguistic units. We see that some phrases are kept: “are playing outdoors” (between Q and C+), “are playing ” (between Q and C−). Some phrases are changed into related semantics on pur- pose: “the young boys” (Q) → “the kids” (C+ & C−), “the man is smiling nearby” (Q) → “near a man with a smile” (C+) or → “an old man is standing in the background” (C−) . We can see that the kept parts have stronger alignments (green color), and changed parts have weaker alignments (blue color). Here, by “strong” / “weak” we mean …

Content truncated to 500 words