Is Character Trigram Overlapping Ratio Still the Best Similarity Measure for Aligning Sentences in a Paraphrased Corpus?

Abstract Sentence alignment is an essential step in studying the mapping among different language expressions, and the character trigram overlapping ratio was reported to be the most effective similarity measure in aligning sentences in the text simplification dataset. However, the appropriateness of each similarity measure depends on the characteristics of the corpus to be aligned. This paper studies if the character trigram is still a suitable similarity measure for the task of aligning sentences in a paragraph paraphrasing corpus. We compare several embedding-based and non-embeddings model-agnostic similarity measures, including those that have not been studied previously. The evaluation is conducted on parallel paragraphs sampled from the Webis-CPC-11 corpus, which is paragraph paraphrasing dataset. Our results a show that modern BERT-based measures such as Sentence-BERT or BERTScore can lead to significant improvement in this task. Keywords: sentence alignment, sentence similarity, sentence embedding 1 Introduction Monolingual text matching is necessary for many downstream applications, such as Paraphrase Identification and Extraction (Qiu et al., 2006), Question Answering (Weiss et al., 2021), Natural Language Inference (MacCartney et al., 2008), and Text Generation (Barzilay and McKeown, 2005). 1 The nearest semantic associates of the verb decide based on the cosine similarity between the word2vec vectors (trained on English Wikipedia) are those verbs such as: choose (0.64), opt (0.62), persuade (0.61), want (0.58), Take the QA task as an example, identifying the text fragments that match the given question within the associated passage is often required for locating the desired answer. However, modern neural-network (NN) approaches to text matching often suffer from certain limitations when two sequences contain considerably different lexicons or diverse grammatical structures (McCoy et al., 2019). For example, when the verb “decide” in the sentence “They decided to go” is nominalized to the noun “decision” in its paraphrase “They made a decision to go”, the popular word embedding similarity approach might fail as the embedding-vectors of “decide” and “decision” are quite different 1 . Another example is a pair of sentences “A cat is chasing a dog.” and “A dog is chasing a cat.”, which contain the same set of lexicons and syntactic structure but with opposite meanings. Furthermore, the NN approaches frequently fail while the matching involves multi-word expressions, or when expressions require compositionality handling (Blevins et al., 2018; Hupkes et al., 2020; Zhou et al., 2020). For example, it is difficult to match expressions “put off” and “procrastinate” using basic word embeddings, as the real meaning of the idiom “put off” is not the sum of the meanings of its tokens. We found that the limitations of NN models in text matching could be greatly alleviated by utilizing lexico-syntactic paraphrasing patterns such as [VP[VBN[see]NP[X1]]][S[NP[X1]VP[VBD[be] refuse (0.57), insist (0.56). However, the noun decision only has a similarity score 0.512, which means that its similarity to the verb decide is even less than that between decide and its quasi-antonymous refuse. Is Character Trigram Overlapping Ratio Still the Best Similarity Measure for Aligning Sentences in a Paraphrased Corpus? Aleksandra Smolka SNHCC TIGP Institute of …

Content truncated to 500 words