Toward Interpretable Semantic Textual Similarity via Optimal Transport-based Contrastive Sentence Learning

Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics Volume 1: Long Papers, pages 5969 - 5979 May 22-27, 2022 c⃝2022 Association for Computational Linguistics Toward Interpretable Semantic Textual Similarity via Optimal Transport-based Contrastive Sentence Learning Seonghyeon Lee1∗, Dongha Lee2, Seongbo Jang1∗, and Hwanjo Yu1† Computer Science and Engineering, POSTECH, Pohang, Republic of Korea1 University of Illinois at Urbana-Champaign (UIUC), Urbana, IL, United States2 {sh0416,jang.sb,hwanjoyu}@postech.ac.kr, donghal@illinois.edu Abstract Recently, finetuning a pretrained language model to capture the similarity between sen- tence embeddings has shown the state-of-the- art performance on the semantic textual sim- ilarity (STS) task. However, the absence of an interpretation method for the sentence sim- ilarity makes it difficult to explain the model output. In this work, we explicitly describe the sentence distance as the weighted sum of contextualized token distances on the basis of a transportation problem, and then present the optimal transport-based distance measure, named RCMD; it identifies and leverages semantically-aligned token pairs. In the end, we propose CLRCMD, a contrastive learn- ing framework that optimizes RCMD of sen- tence pairs, which enhances the quality of sen- tence similarity and their interpretation. Exten- sive experiments demonstrate that our learn- ing framework outperforms other baselines on both STS and interpretable-STS benchmarks, indicating that it computes effective sentence similarity and also provides interpretation con- sistent with human judgement. The code and checkpoint are publicly available at https: //github.com/sh0416/clrcmd. 1 Introduction Predicting the semantic similarity between two sen- tences has been extensively studied in the litera- ture (Gomaa et al., 2013; Agirre et al., 2015; Ma- jumder et al., 2016; Cer et al., 2017). Several recent studies successfully utilized a pretrained language model such as BERT (Devlin et al., 2019) by fine- tuning it to capture sentence similarity (Reimers and Gurevych, 2019). To be specific, they define a similarity score between sentence embeddings, which are obtained by aggregating contextualized token embeddings (e.g., avg pooling) or using a spe- cial token (e.g., [CLS]), then optimize the score ∗This work was done during internship at Scatterlab. †Corresponding author Figure 1: An illustrative example of a transportation problem in contextualized embedding space. The ex- isting distance measure between average-pooled sen- tence embeddings (orange) cannot clearly capture the distances of semantically-aligned token pairs (blue). for natural language inference (NLI) or semantic textual similarity (STS) tasks (Gao et al., 2021). Along with the quality of sentence similarity, in- terpreting the predicted sentence similarity is also important for end-users to better understand the results (Agirre et al., 2016; Gilpin et al., 2018; Rogers et al., 2020). In general, finding out the cross-sentence alignment and the importance of each aligned part is useful for analyzing sentence similarity (Sultan et al., 2015). For example, there were several attempts to use explicit features (e.g., TF-IDF) for easily analyzing the interaction among the shared terms (Salton and Buckley, 1988) or to adopt sophisticated metrics (e.g., word mover’s dis- tance) for explicitly describing it by the importance and similarity of word pairs across two sentences (Kusner et al., …

Content truncated to 500 words