ASOBEK at SemEval-2016 Task 1: Sentence Representation with Character N-gram Embeddings for Semantic Textual Similarity

Proceedings of SemEval-2016, pages 736–740, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics ASOBEK at SemEval-2016 Task 1: Sentence Representation with Character N-gram Embeddings for Semantic Textual Similarity Asli Eyecioglu and Bill Keller University of Sussex Department of Informatics Brighton, BN19QJ, UK A.Eyecioglu@sussex.ac.uk, billk@sussex.ac.uk Abstract A growing body of research has recently been conducted on semantic textual similarity using a variety of neural network models. While re- cent research focuses on word-based represen- tation for phrases, sentences and even paragraphs, this study considers an alternative approach based on character n-grams. We generate embeddings for character n-grams using a continuous-bag-of-n-grams neural network model. Three different sentence rep- resentations based on n-gram embeddings are considered. Results are reported for experi- ments with bigram, trigram and 4-gram em- beddings on the STS Core dataset for SemEval-2016 Task 1. 1 Introduction This paper presents an approach for finding the degree of semantic similarity between sentence pairs. Semantic textual similarity (STS) is of rele- vance to many NLP applications. Recent tasks in recognizing textual entailment, sentence comple- tion and paraphrase identification are closely relat- ed. The approach described here makes use of a neural network (NN) algorithm (word2vec) that is typically used to generate word embeddings (Mikolov et al., 2013a). Rather than generating vector representations for words however, we pro- pose a character n-gram-to-vector approach. A sen- tence is then represented as a vector generated through a combination of character n-gram embed- dings. The use of character level vectors has been pro- posed in a number of recent studies. Subword lan- guage models that use the combination of characters, syllables and frequent words have been explored by Mikolov et al. (2012). Character-level language modeling has been performed for model- ing OOV words, where using words as the atomic units of the model would not be sufficient to assign a probability score. In Ling et al. (2015), word rep- resentations are composed of vectors of characters, called character to word (C2W). The C2W vectors are used successfully for language modeling and POS tagging without any handcrafted features. The resulting model is competitive on English POS tagging with the Stanford POS tagger word look- up tables and also produces a notable improvement in results for morphologically rich languages such as Turkish. Kim et al. (2015) apply a simple con- volutional neural network model, which uses char- acter level inputs for word representations. Again, this method outperforms the models that use word/morpheme level features in morphologically rich languages, while also having competitive re- sults in English. Huang et al. (2013) introduce a word hashing technique using character n-grams to scale up training of deep NN models for large- scale web search applications. Our motivation for exploring character n-grams derives in part from previous work we have con- ducted on paraphrase identification (PI). The PI task is that of deciding whether two sentences have the same meaning. We have shown that sentence representations based on bags or sets of character n-gram features can perform well at …

Content truncated to 500 words