MayoClinicNLP–CORE: Semantic representations for textual similarity

Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 1: Proceedings of the Main Conference and the Shared Task, pages 148–154, Atlanta, Georgia, June 13-14, 2013. c⃝2013 Association for Computational Linguistics MayoClinicNLP–CORE: Semantic representations for textual similarity Stephen Wu Mayo Clinic Rochester, MN 55905 wu.stephen@mayo.edu Dongqing Zhu & Ben Carterette University of Delaware Newark, DE 19716 {zhu,carteret}@cis.udel.edu Hongfang Liu Mayo Clinic Rochester, MN 55905 liu.hongfang@mayo.edu Abstract The Semantic Textual Similarity (STS) task examines semantic similarity at a sentence- level. We explored three representations of semantics (implicit or explicit): named enti- ties, semantic vectors, and structured vectorial semantics. From a DKPro baseline, we also performed feature selection and used source- specific linear regression models to combine our features. Our systems placed 5th, 6th, and 8th among 90 submitted systems. 1 Introduction The Semantic Textual Similarity (STS) task (Agirre et al., 2012; Agirre et al., 2013) examines semantic similarity at a sentence-level. While much work has compared the semantics of terms, concepts, or doc- uments, this space has been relatively unexplored. The 2013 STS task provided sentence pairs and a 0–5 human rating of their similarity, with training data from 5 sources and test data from 4 sources. We sought to explore and evaluate the usefulness of several semantic representations that have had recent significance in research or practice. First, information extraction (IE) methods often implic- itly consider named entities as ad hoc semantic rep- resentations, for example, in the clinical domain. Therefore, we sought to evaluate similarity based on named entity-based features. Second, in many appli- cations, an effective means of incorporating distri- butional semantics is Random Indexing (RI). Thus we consider three different representations possi- ble within Random Indexing (Kanerva et al., 2000; Sahlgren, 2005). Finally, because compositional distributional semantics is an important research topic (Mitchell and Lapata, 2008; Erk and Pad´o, 2008), we sought to evaluate a principled compo- sition strategy: structured vectorial semantics (Wu and Schuler, 2011). The remainder of this paper proceeds as follows. Section 2 overviews our similarity metrics, and Sec- tion 3 overviews the systems that were defined on these metrics. Competition results and additional analyses are in Section 4. We end with discussion on the results in Section 5. 2 Similarity measures Because we expect semantic similarity to be multi- layered, we expect that we will need many similar- ity measures to approximate human similarity judg- ments. Rather than reinvent the wheel, we have cho- sen to introduce features that complement existing successful feature sets. We utilized 17 features from DKPro Similarity and 21 features from TakeLab, i.e., the two top-performing systems in the 2012 STS task, as a solid baseline. These are summarized in Table 1. We introduce 3 categories of new similarity metrics, 9 metrics in all. 2.1 Named entity measures Named entity recognition provides a common ap- proximation of semantic content for the informa- tion extraction perspective. We define three simple similarity metrics based on named entities. First, we computed the named entity overlap (exact string matches) between the two …

Content truncated to 500 words