AspectCSE: Sentence Embeddings for Aspect-Based Semantic Textual Similarity Using Contrastive Learning and Structured Knowledge

Proceedings of Recent Advances in Natural Language Processing, pages 1054–1065 Varna, Sep 4–6, 2023 https://doi.org/10.26615/978-954-452-092-2_113 1054 AspectCSE: Sentence Embeddings for Aspect-based Semantic Textual Similarity Using Contrastive Learning and Structured Knowledge Tim Schopf1, Emanuel Gerber1, Malte Ostendorff2, and Florian Matthes1 1Technical University of Munich, Department of Computer Science, Garching, Germany 2DFKI GmbH, Berlin, Germany {tim.schopf,emanuel.gerber,matthes}@tum.de malte.ostendorff@dfki.de Abstract Generic sentence embeddings provide a coarse- grained approximation of semantic textual simi- larity but ignore specific aspects that make texts similar. Conversely, aspect-based sentence em- beddings provide similarities between texts based on certain predefined aspects. Thus, sim- ilarity predictions of texts are more targeted to specific requirements and more easily explain- able. In this paper, we present AspectCSE, an approach for aspect-based contrastive learning of sentence embeddings. Results indicate that AspectCSE achieves an average improvement of 3.97% on information retrieval tasks across multiple aspects compared to the previous best results. We also propose using Wikidata knowl- edge graph properties to train models of multi- aspect sentence embeddings in which multi- ple specific aspects are simultaneously consid- ered during similarity predictions. We demon- strate that multi-aspect embeddings outperform single-aspect embeddings on aspect-specific in- formation retrieval tasks. Finally, we exam- ine the aspect-based sentence embedding space and demonstrate that embeddings of semanti- cally similar aspect labels are often close, even without explicit similarity training between dif- ferent aspect labels. 1 Introduction Sentence embeddings are representations of sen- tences or short text paragraphs in a dense vec- tor space, such that similar sentences are close to each other (Reimers and Gurevych, 2020). Learn- ing sentence embeddings is a fundamental task in natural language processing (NLP) and has al- ready been extensively investigated in the litera- ture (Kiros et al., 2015; Hill et al., 2016; Conneau et al., 2017; Logeswaran and Lee, 2018; Cer et al., 2018; Reimers and Gurevych, 2019; Gao et al., 2021; Schopf et al., 2023d). Generic sentence em- beddings can be used to distinguish between sim- ilar and dissimilar sentences, without considering which aspects of sentences are similar (Ostendorff et al., 2020a). Moreover, they are often evaluated on generic semantic textual similarity (STS) tasks (Marelli et al., 2014; Agirre et al., 2012, 2013, 2014, 2015, 2016; Cer et al., 2017) in which sentence sim- ilarity scores rely on human annotations. However, the concept of generic STS is not well defined, and text similarity depends heavily on the aspects that make them similar (B¨ar et al., 2011; Ostendorff et al., 2020b, 2022). We follow the argument of B¨ar et al. (2011) on textual similarity and define aspects as inherent properties of texts that must be considered when predicting their semantic similar- ity. Based on the different aspects focused on in texts, their similarities can be perceived very dif- ferently. Figure 1 illustrates an example of aspect- based STS. For example, Wikipedia introduction texts of famous individuals can generally be consid- ered similar as all texts introduce people who are known to the public. However, focusing the com- parison on specific aspects (e.g., country of birth …

Content truncated to 500 words