Voltron: A Hybrid System For Answer Validation Based On Lexical And Distance Features

Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 242–246, Denver, Colorado, June 4-5, 2015. c⃝2015 Association for Computational Linguistics Voltron: A Hybrid System For Answer Validation Based On Lexical And Distance Features Ivan Zamanov1, Nelly Hateva1, Marina Kraeva1, Ivana Yovcheva1, Ivelina Nikolova2, Galia Angelova2 1 FMI, Sofia University, Sofia, Bulgaria 2 IICT, Bulgarian Academy of Sciences, Sofia, Bulgaria ivo.zamanov@gmail.com, nelly.hateva@gmail.com, mvkraeva@gmail.com, ivana.yovcheva@gmail.com, iva@lml.bas.bg, galia@lml.bas.bg Abstract The purpose of this paper is to describe our submission to the SemEval-2015 Task 3 on Answer Selection in Community Question Answering. We participated in subtask A, where the systems had to classify community answers for a given question as definitely rel- evant, potentially useful, or irrelevant. For ev- ery question-answer pair in the training data we extract a vector with a variety of features. These vectors are then fed to a MaxEnt classi- fier for training. Given a question and an an- swer the trained classifier outputs class proba- bilities for each of the three desired categories. The one with the highest probability is cho- sen. Our system scores better than the average score in subtask A of Task 3. 1 Introduction Nowadays, text analysis and semantic similarity are subject to a lot of research and experiments due to the growth of social media influence, the increas- ing usage of forums for finding a solution of com- mon known problems and the Web upgrowth. As beginners in the computational linguistics field, we were very interested in dealing with these topics and have found Answer Validation as a good start. Our team chose to focus on subtask A of Task 3 in the SemEval-2015 workshop, namely Answer selection in community question answering data. In order to achieve good results, we combined most of the techniques familiar to us. We process the data as question-answer pairs. The framework GATE (Cun- ningham et al., 2002) was used for the preprocess- ing in the system because it offers convenient natu- ral language processing pipelines and has an API al- lowing for system integration. For classification we used the Maximum Entropy classifier provided by MALLET (McCallum and Kachites, 2002). We use a combination of surface, morphological, syntactic, and contextual features as well as distance metrics between the question and answer. Distance metrics are based on word2vec (Mikolov et al., 2013a) and DKPro Similarity (B¨ar, et al.), (de Castilho, 2014). 2 Related work Several recent systems were created and used for similar analysis. Although their applications have some differences from the system described in this paper, we consider them relevant because they deal with semantic similarity. (Bas¸kaya, 2014) uses Vector Space Models which have some similarity to our usage of word2vec cen- troid metrics with the difference that we do not orga- nize the whole text according to the structure of the result matrix, as the VSMs do. The cosine similarity is common for both systems. The big difference is that we use only the input words while in his system the words’ likely synonyms according …

Content truncated to 500 words