UINSUSKA-TiTech at SemEval-2017 Task 3: Exploiting Word Importance Levels for Similarity Features for CQA

Proceedings of the 11th International Workshop on Semantic Evaluations (SemEval-2017), pages 370–374, Vancouver, Canada, August 3 - 4, 2017. c⃝2017 Association for Computational Linguistics UINSUSKA-TiTech at SemEval-2017 Task 3: Exploiting Word Importance Levels for Similarity Features for CQA Surya Agustian1,2 and Hiroya Takamura2 1Teknik Informatika, UIN Sultan Syarif Kasim Riau, Indonesia 1Computational Intelligence & System Science, Tokyo Institute of Technology, Japan 2Institute of Innovative Research, Tokyo Institute of Technology, Japan sagustian{@uin-suska.ac.id, @lr.pi.titech.ac.jp}, takamura@pi.titech.ac.jp Abstract The majority of core techniques to solve many problems in Community Question Answering (CQA) task rely on similarity computation. This work focuses on simi- larity between two sentences (or questions in subtask B) based on word embeddings. We exploit words importance levels in sentences or questions for similarity fea- tures, for classification and ranking with machine learning. Using only 2 types of similarity metric, our proposed method has shown comparable results with other com- plex systems. This method on subtask B 2017 dataset is ranked on position 7 out of 13 participants. Evaluation on 2016 da- taset is on position 8 of 12, outperforms some complex systems. Further, this find- ing is explorable and potential to be used as baseline and extensible for many tasks in CQA and other textual similarity based system. 1 Introduction Community Question Answering (CQA) is get- ting popular for requesting valid information from experienced people. However, waiting for such favorable answers for a new submitted ques- tion, is a boring task for users once querying to online community forums. IR system can utilize thread in online community forum for question queries. Even so, the appropriate answers are of- ten mixed among snippets of many irrelevant documents, and opening full articles is still re- quired. A post-processing system is needed in or- der to obtain the most relevant answers. CQA tasks want to address this need, to help user get the most favorable answers by improving IR sys- tem results. SemEval CQA Task 3 is designed to gather some possible solutions, in five coherent subtasks (Nakov et al., 2017). Since some subtasks are re- lated, we focus only on subtask B, with goal to provide a good basis framework for solving prob- lem in other subtasks. In Task 3 of the previous year, word embeddings obtained with a tool such as word2vec (Mikolov et al., 2013, 2013b) contrib- uted to the best systems for all subtasks. In addi- tion, machine learning based methods were most- ly ranked in the top positions for all subtasks. The most popular machine learning approach was SVM for classification, regression and ranking, while neural networks, even though widely used, did not win any subtasks (Nakov et al., 2016). Most machine learning approaches rely on sev- eral similarity features as the basis. Various tech- niques to compute semantic similarity based on word embeddings, were used by Franco-Salvador et al. (2016), Filice et al. (2016), Mohtarami et al. (2016), Wu and Lan (2016), and Mihaylov and Nakov (2016). Besides, they also used various lexical and semantic similarities including simple …

Content truncated to 500 words