NAI-SEA at SemEval-2018 Task 5: An Event Search System

Proceedings of the 12th International Workshop on Semantic Evaluation (SemEval-2018), pages 674–678 New Orleans, Louisiana, June 5–6, 2018. ©2018 Association for Computational Linguistics NAI-SEA at SemEval-2018 Task 5: An Event Search System Yingchi Liu, Quanzhi Li and Luo Si Alibaba Group, Inc Bellevue, WA 98004, USA fyingchi.liu,quanzhi.li,luo.sig@alibaba-inc.com Abstract In this paper, we describe Alibaba’s participating system in the semEval-2018 Task5: Counting Events and Participants in the Long Tail. We designed and implemented a pipeline system that consists of components to extract question properties and document features, document event category classifications, document retrieval and document clustering. To retrieve the majority of the relevant documents, we carefully designed our system to extract key information from each question and document pair. After retrieval, we perform further document clustering to count the number of events. The task contains 3 subtasks, on which we achieved F1 score of 78.33, 50.52, 63.59 , respectively, for document level retrieval. Our system ranks first in all the three subtasks on document level retrieval, and it also ranks first in incident-level evaluation by RSME measure in subtask 3. 1 Introduction In this paper we present our system developed for participanting in the semEval-2018 Task5: Counting Events and Participants in the Long Tail. Given a set of questions and a corpus of documents mainly from news articles, the system needs to provide a numeric answer together with the supporting documents that directly relate to the answer (Postma et al., 2018). According to the official task rule, participants can also optionally provide the text mentions of events in the documents, but we did not participate this year. The task contains 3 subtasks. Subtask 1 is to retrieve all the relevant documents related to one single event asked in the question. Subtask 2 and subtask 3 require the system not only retrieve relevant documents, but also count the number of events or number of participants. Event detection and extraction has been intensively studied (Choubey and Huang, 2017; Nguyen et al., 2016a,b; Nguyen and Grishman, 2016; Feng et al., 2016; Ji and Grishman, 2008). Most of those research used a corpus of annotated documents for training. In this task, annotated documents were not provided, but the key factors to retrieve relevant documents are provided by the questions. Therefore, our system starts from a document retrieval system via key information extraction and matching and follows a document clustering component. 2 System Description We developed a pipeline system for the task, including question parsing, document feature generation, document event type classifications, document retrieval and document clustering. 2.1 Question Properties In this task, each question contains three components: the event type and two event properties. The two event properties provided are either the time, the location or the participant of the event. And specifications for these properties can vary in granularity (e.g. day/month/year, city/state, first/last/full name). Details can refer to official task description (Postma et al., 2018). In this task, we consider four event types (i.e. killing, injuring, fire burning, job firing). But in training data, …

Content truncated to 500 words