IISCNLP at SemEval-2016 Task 2: Interpretable STS with ILP based Multiple Chunk Aligner

Proceedings of SemEval-2016, pages 790–795, San Diego, California, June 16-17, 2016. c⃝2016 Association for Computational Linguistics IISCNLP at SemEval-2016 Task 2: Interpretable STS with ILP based Multiple Chunk Aligner Lavanya Sita Tekumalla and Sharmistha lavanya.tekumalla@gmail.com sharmistha.jat@gmail.com Indian Institute of Science, Bangalore, India Abstract Interpretable semantic textual similarity (iSTS) task adds a crucial explanatory layer to pairwise sentence similarity. We address various components of this task: chunk level semantic alignment along with assignment of similarity type and score for aligned chunks with a novel system presented in this paper. We propose an algorithm, iMATCH, for the alignment of multiple non-contiguous chunks based on Integer Linear Programming (ILP). Similarity type and score assignment for pairs of chunks is done using a supervised multiclass classification technique based on Random Forrest Classifier. Results show that our algorithm iMATCH has low execution time and outperforms most other participating systems in terms of alignment score. Of the three datasets, we are top ranked for answer- students dataset in terms of overall score and have top alignment score for headlines dataset in the gold chunks track. 1 Introduction and Related Work Semantic Textual Similarity (STS) refers to measur- ing the degree of equivalence in underlying seman- tics(meaning) of a pair of text snippets. It finds ap- plications in information retrieval, question answer- ing and other natural language processing tasks. In- terpretable STS (iSTS) adds an explanatory layer, by measuring similarity across chunks of segmented text, leading to an improved interpretability. It in- volves aligning multiple chunks across sentences with similar meaning along with similarity score(0- 5) and type assignment. Interpretable STS task was first introduced as a pilot task in 2015 Semeval STS task. Several ap- proaches were proposed including NeRoSim (Ban- jade et al., 2015), UBC-Cubes (Agirre et al., 2015) and Exb-Thermis (H¨anig et al., 2015). For the task of alignment, these submissions used approaches based on monolingual aligner using word similarity and contextual features (Md Arafat Sultan and Sum- mer, 2014), JACANA that uses phrase based semi- markov CRFs (Yao and Durme, 2015) and Hun- garian Munkers algorithm (Kuhn and Yaw, 1955). Other popular approaches for mono-lingual align- ment include two-stage logistic-regression based aligner (Md Arafat Sultan and Summer, 2015), tech- niques based on edit rate computation such as (lien Maxe Anne Vilnat, 2011) and TER-Plus (Snover et al., 2009). (Bodrumlu et al., 2009) used ILP for word alignment problem. The iSTS task in 2016 introduced problem of many-to-many chunk align- ment, where multiple non-contiguous chunks of the source can align with multiple-non-contiguous chunks of the target sentence, that previous mono- lingual alignment techniques cannot handle. We propose iMATCH, a new technique for monolingual alignment for many-to-many alignment at the chunk level, that can combine non-contiguous chunks based on integer linear programming (ILP). We also explore several features to define a similarity score between chunks to define the objective function for our optimization problem, similarity type and score classification modules. To summarize our contribu- tions: • We propose a novel algorithm for monolingual alignment : …

Content truncated to 500 words