iUBC at SemEval-2016 Task 2: RNNs and LSTMs for interpretable STS

Proceedings of SemEval-2016, pages 771–776, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics iUBC at SemEval-2016 Task 2: RNNs and LSTMs for interpretable STS I˜nigo Lopez-Gazpio and Eneko Agirre and Montse Maritxalar IXA NLP group. University of the Basque Country (UPV/EHU) Manuel Lardizabal 1, 20.018, Donostia, Basque Country inigo.lopez@ehu.eus Abstract This paper describes iUBC, a neural network based approach that achieves competitive re- sults on the interpretable STS task (iSTS 2016). Actually, it achieves top performance in one of the three datasets. iUBC makes use of a jointly trained classifier and regressor, and both models work on top of a recurrent neural network. Through the paper we provide de- tailed description of the approach, as well as the results obtained in iSTS 2015 test, iSTS 2016 training and iSTS 2016 test. 1 Introduction Semantic Textual Similarity (STS) aims to catch the degree of equivalence between a pair of text nuggets. Interpretable STS (iSTS) is beyond STS in that it adds fine-grained information when evaluating the equivalence between text snippets. This explanatory layer is achieved by aligning text segments pertain- ing to one sentence with the segments pertaining to the second sentence, and, for each alignment, indi- cating a relation label and a similarity score. In sum, alignments consist of a similarity score and a relation label that are defined as follows. On the one hand, the relation label has to be chosen from a set of categorical values (equivalence, opposition, specialization, similarity and other kind of relation). On the other hand, the similarity score has to be a real number bounded by (0,5]. Apart from this, there is an extra label to handle not aligned text segments. The present paper describes iUBC and its par- ticipation in the International Workshop on Seman- tic Evaluation (SemEval-2016) task 2: Interpretable Semantic Textual Similarity. To check the task in full detail please refer to Agirre et al. (2016). Note that some of the authors participated in the organi- zation of the task. Organizers prevented developers from access to the test dataset, and only allowed to access the same data as the rest of participants. The paper is organized as follows. Section 2 de- scribes iUBC’s components, section 3 describes de- velopment performance and run configurations, sec- tion 4 shows the results obtained in the iSTS 2016 task, and, finally, section 5 mentions the conclusions and future work directions. 2 System Description iUBC is composed of three components. The first component, Input handling and chunking (section 2.1), is responsible for reading the input and iden- tifying segments over sentences; the second compo- nent, Alignment (section 2.2), takes care of aligning segments; and, finally, the third component, Joint classification and scoring (section 2.3), handles the assignment of similarity scores and relation labels. The main contribution of this architecture resides in the third component, in which a classifier and a re- gressor have been trained jointly on top of a recur- rent artificial neural network (ANN). 2.1 Input handling and chunking The iSTS …

Content truncated to 500 words