Enhancing Unsupervised Sentence Similarity Methods with Deep Contextualised Word Representations

Proceedings of Recent Advances in Natural Language Processing, pages 994–1003, Varna, Bulgaria, Sep 2–4, 2019. https://doi.org/10.26615/978-954-452-056-4_115 994 Enhancing Unsupervised Sentence Similarity Methods with Deep Contextualised Word Representations Tharindu Ranasinghe, Constantin Or˘asan and Ruslan Mitkov Research Group in Computational Linguistics University of Wolverhampton, UK {t.d.ranasinghehettiarachchige, c.orasan, r.mitkov }@wlv.ac.uk Abstract Calculating Semantic Textual Similarity (STS) plays a significant role in many applications such as question answering, document summarisation, information re- trieval and information extraction. All modern state of the art STS methods rely on word embeddings one way or an- other. The recently introduced contextu- alised word embeddings have proved more effective than standard word embeddings in many natural language processing tasks. This paper evaluates the impact of several contextualised word embeddings on un- supervised STS methods and compares it with the existing supervised/unsupervised STS methods for different datasets in dif- ferent languages and different domains. 1 Introduction Measuring Semantic Textual Similarity (STS) is calculating the degree of semantic equivalence be- tween two snippets of text (Agirre et al., 2016). Earlier, STS tasks largely focused on similarity between short texts such as abstracts and prod- uct descriptions (Li et al., 2006; Mihalcea et al., 2006). Recently, STS tasks at the International Workshops on Semantic Evaluation (SemEval) fo- cused on measuring STS between full sentence pairs. The introduction of competitive STS tasks led to the development of standard datasets like the SICK corpus (Bentivogli et al., 2016) and stan- dardised the similarity score as a numerical value between 1 and 5 (Agirre et al., 2014). Having a good STS metric is crucial for many natural language processing applications such as information retrieval (IR) (Majumder et al., 2016), text summarisation (Aliguliyev, 2009; Steinberger and Jezek, 2004), question answering (Mohler et al., 2011) and text classification (Rocchio, 1971). Semantic similarity also contributes to many semantic web applications like community extraction, ontology generation and entity disam- biguation (Li et al., 2006), and it is also useful for Twitter search (Salton et al., 1997), where it is re- quired to accurately measure semantic relatedness between concepts or entities (Xu et al., 2015). STS is not limited only to natural language processing. For example in Biomedical Informatics, it can be used to compare genes (Ferreira and Couto, 2010). Given the growing importance of having a good STS metric and as a result of the SemEval work- shops, researchers have proposed numerous STS methods. Most of the early approaches were based on traditional machine learning and in- volved heavy feature engineering (B´echara et al., 2015). With the advances of word embeddings, and as a result of the success neural networks have achieved in other fields, most of the methods pro- posed in recent years rely on neural architectures (Tai et al., 2015; Shao, 2017). Neural networks are preferred over traditional machine learning mod- els as they generally tend to perform better than traditional machine learning models. They also do not rely on explicit linguistics features which have to be extracted before the ML model is learnt. De- termining the …

Content truncated to 500 words