Vanilla Recurrent Neural Networks for Interpretable Semantic Textual Similarity

Vanilla Recurrent Neural Networks for Interpretable Semantic Textual Similarity Piotr Andruszkiewicz Warsaw University of Technology piotr.andruszkiewicz@pw.edu.pl Barbara Rychalska Warsaw University of Technology b.rychalska@mini.pw.edu.pl Abstract Semantic similarity systems assess to what ex- tent two words, phrases, sentences are simi- lar in their meaning. In this task, rule-based and neural network systems achieve the best results. The former requires intensive human workload and the latter needs heavy comput- ing. Can we achieve high accuracy without hand-crafted rules and without intensive com- puting? In this paper we present three types of Vanilla Recurrent Neural Networks that ful- fill the aforementioned requirements for In- terpretable Semantic Textual Similarity task and we compare them to the systems from Se- mEval competition. 1 Introduction Semantic similarity, also called paraphrase detec- tion, focuses on similarity of words, phrases, sen- tences in terms of semantic equivalence. Having a different grammar construction or different words, yet synonyms, used, two sentences may convey the same meaning. Semantic similarity assesses the level of semantic equivalence, e.g., by saying that a pair of sentences is not semantically equivalent or is equivalent to some extent. There are many different specifications of seman- tic similarity task. We focus on two definitions of this task used in SemEval competition (Cer et al., 2015). The first one is a basic approach, called herein basic semantic similarity task, that expresses semantic similarity as one number from 0 to 5. 0 means that there is no semantic similarity, e.g., A woman is slicing big pepper. vs. A dog is moving its mouth. 5 is assigned when there is perfect semantic equivalence, e.g., The man cut down a tree with an axe. vs. A man chops down a tree with an axe. In the middle there are pairs that are similar to some extent, e.g., a pair People are playing cricket. vs. Men are playing cricket. is scored as 3. A more complicated specification of a task in question, called Interpretable Semantic Textual Sim- ilarity (iSTS), defines 8 alignment types between chunks of sentences, e.g., EQUI - semantic equiva- lence – in Olympics, at Olympics, OPPO - semantic opposition – lower vs. higher. Additionally, simi- larity for each alignment is scored on a scale 0-5. In this paper, we focus on Interpretable Semantic Tex- tual Similarity (iSTS) task as it gives more insight in the justification of the similarity. We can distinguish two main types of systems that assess semantic similarity, namely; hand-crafted rule-based systems and statistical systems. The first group uses many rules prepared by linguists and is highly customized for a specific task. Such systems need a lot of manual work. Statistical systems use trained models to assess semantic similarity. Some of them also involve lots of manual work in features engineering. We focus on statistical systems that do not require manual work and here comes deep learn- ing approach. Moreover, we prefer network architectures pow- ered by basic network cell, because amount of anno- tated data is limited for Interpretable Semantic Tex- tual Similarity (iSTS). Hence, …

Content truncated to 500 words