FEUP at SemEval-2018 Task 5: An Experimental Study of a Question Answering System

Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 667–673 New Orleans, Louisiana, June 5–6, 2018. ©2018 Association for Computational Linguistics FEUP at SemEval-2018 Task 5: An Experimental Study of a Question Answering System Carla Abreu Faculdade Engenharia Universidade do Porto ei08165@fe.up.pt Eug´enio Oliveira Faculdade Engenharia Universidade do Porto LIACC eco@fe.up.pt Abstract We present the approach developed at the Fac- ulty of Engineering of the University of Porto to participate in SemEval-2018 Task 5: Count- ing Events and Participants within Highly Am- biguous Data covering a very long tail.1 The work described here presents the experimental system developed to extract entities from news articles for the sake of Question Answering. We propose a supervised learning approach to enable the recognition of two different types of entities: Locations and Participants. We also discuss the use of distance-based algorithms (using Levenshtein distance and Q-grams) for the detection of documents’ closeness based on the entities extracted. For the experiments, we also used a multi-agent system that im- proved the performance. 1 Introdution Thousands of news articles are published every day on several media outlets. Representing and reasoning over all events in these articles is a chal- lenging task. For instance, if we would like to answer questions about these articles like: How many people died on the shootings in Philippi in 30th September, 2017? or How many people died last year on Birmingham? or How many people were killed by John List?, a deep understanding is needed of many phenomena in the articles. For example, news story updates and duplicate news need to be considered in the answer processing. We can simplify the problem by identifying rele- vant elements from the news entities and create a structured representation to store these data. Named Entity Recognition (NER) is a task that aims at identifying and classifying entity mentions in free text. Message Understanding Conference (MUC) defines the entities as belonging to three 1https://competitions.codalab.org/competitions/17285 categories:2 1. Enamex: names, such as Locations, Persons, Organizations, and others 2. Timex: tem- poral expressions 3. Numex: numerical elements, such as numbers and percentages. In this paper, we present an experimental study to extract entities from news articles to answer questions. We make use of a supervised learn- ing approach to deal with the recognition of two different kind of entities: Locations (e.g. Philippi, Birmingham) and Participants (e.g. John List). We also have studied the use of distance algorithms (Levenshtein and Q-grams) for the near document detection based on entities extracted. The remainder of the paper is organized as fol- lows. In Section 2, we describe SemEval-2018 Task 5, followed by an overview of the state of the art in Named Entity Recognition in Section 3. In Section 4, we present the state of the art in the Near Document Detection task, followed by the description of the system architecture in Section 5. In Section 6, we presents the approach, fol- lowed by the experimental setup in Section 7. The results are discussed in Section 8. 2 Task …

Content truncated to 500 words