Towards Structure-aware Paraphrase Identification with Phrase Alignment Using Sentence Encoders

Proceedings of the 29th International Conference on Computational Linguistics, pages 4113–4123 October 12–17, 2022. 4113 Towards Structure-aware Paraphrase Identification with Phrase Alignment Using Sentence Encoders Qiwei Peng David Weir Julie Weeds University of Sussex Brighton, UK {qiwei.peng, d.j.weir, j.e.weeds}@sussex.ac.uk Abstract Previous works have demonstrated the effec- tiveness of utilising pre-trained sentence en- coders based on their sentence representations for meaning comparison tasks. Though such representations are shown to capture hidden syntax structures, the direct similarity compari- son between them exhibits weak sensitivity to word order and structural differences in given sentences. A single similarity score further makes the comparison process hard to inter- pret. Therefore, we here propose to combine sentence encoders with an alignment compo- nent by representing each sentence as a list of predicate-argument spans (where their span representations are derived from sentence en- coders), and decomposing the sentence-level meaning comparison into the alignment be- tween their spans for paraphrase identification tasks. Empirical results show that the align- ment component brings in both improved per- formance and interpretability for various sen- tence encoders. After closer investigation, the proposed approach indicates increased sensitiv- ity to structural difference and enhanced ability to distinguish non-paraphrases with high lexi- cal overlap. 1 Introduction Sentence meaning comparison measures the seman- tic similarity of two sentences. Specifically, the task of paraphrase identification binarises the simi- larity as paraphrase or non-paraphrase depending on whether they express similar meanings (Bhagat and Hovy, 2013). This task benefits many natural language understanding applications, like plagia- rism identification (Chitra and Rajkumar, 2016) and fact checking (Jiang et al., 2020), where it is important to detect same things said in different ways. The difference in sentence structures is impor- tant for distinguishing their meanings. However, as shown in Table 1 and 3, many existing paraphrase identification datasets exhibit high correlation be- tween positive pairs and the degree of their lexi- cal overlap, such as the Microsoft Research Para- phrase Corpus (MSRP) (Dolan and Brockett, 2005). Models trained on them tend to mark sentence pairs with high word overlap as paraphrases despite clear clashes in meaning. In light of this, Zhang et al. (2019b) utilised word scrambling and back translation to create the Paraphrase Adversaries from Word Scrambling (PAWS) datasets which are mainly concerned with word order and structure by creating paraphrase and non-paraphrase pairs with high lexical overlap. As also shown in these two tables, sentence pairs in the PAWS datasets demon- strate much higher lexical overlap and lower corre- lation, which requires models to pay more attention to word order and sentence structure to successfully distinguish non-paraphrases from paraphrases. Recently, various pre-trained sentence encoders have been proposed to produce high-quality sen- tence embeddings for downstream usages (Reimers and Gurevych, 2019; Thakur et al., 2021; Gao et al., 2021). Such embeddings are compared to derive a similarity score for different meaning comparison tasks, including paraphrase identification. Though widely used, sentence encoders still face challenges from different aspects in case of meaning compar- ison. Pre-trained models are observed to capture structural …

Content truncated to 500 words