Overview of the 2014 ALTA Shared Task: Identifying Expressions of Locations in Tweets

Overview of the 2014 ALTA Shared Task: Identifying Expressions of Locations in Tweets Diego Molla Macquarie University, Sydney, Australia diego.molla-aliod@mq.edu.au Sarvnaz Karimi CSIRO, Australia sarvnaz.karimi@csiro.au Abstract This year was the fifth in the ALTA se- ries of shared tasks. The topic of the 2014 ALTA shared task was to identify location information in tweets. As in past com- petitions, we used Kaggle in Class as the framework for submission, evaluation and communication with the participants. In this paper we describe the details of the shared task, evaluation method, and results of the participating systems. 1 Introduction Locations are important pieces of information in social media. When people discuss an event, often they mention where that event is taking place or what they see where. In the case of emergencies, such locations could lead the right resources to the correct place or from the correct route. Also, rec- ommender systems that locate suitable products and services for users require location informa- tion. This year, the fifth Australasian Language Technology Association (ALTA) shared task was set to identify expressions of locations identifiable on the map from Twitter messages. A total of 7 teams registered to the competition, with 4 teams submitting their results. In this paper we describe the background of the shared task, evaluation methods, and results of participating systems. Section 2 describes the shared task. Section 3 gives a short survey of re- lated research. Section 4 describes the data set that was used. Section 5 details the evaluation process. Section 6 shows the results. Section 7 discusses some of the key challenges and issues encountered during the organisation of the shared task. Finally, section 8 concludes the paper and points to the methods used by participating teams. Tweet Location France and Germany join the US and UK in advising their nationals in Libya to leave immediately http://bbc.in/1rVmrDJ France, Germany, US, UK, Libya Dutch investigators not going to MH17 crash site in eastern Ukraine due to security concerns, OSCE monitors say MH17 crash site, eastern Ukraine Seeing early signs of potential flash flooding with stationary storms near St. Marys, Tavistock, Cambridge #onstorm pic.twitter.com/BtogIxgQ5G St. Marys, Tavistock, Cambridge Table 1: Example tweets and their location words. 2 The 2014 ALTA Shared Task The goal of the 2014 shared task was to identify all mentions of locations in the text of tweets. Lo- cation was defined as any specific mention of a country, city, suburb, street, or POI (Point of In- terest). Examples of POI include the name of a shopping centre, such as “Macquarie Centre” or the name of a hospital, e.g., “Ryde Hospital”. This information extraction task is important for appli- cations that attempt to find out where people are or whether they are talking about which location. The shared task required the participants to only identify which word in the text of a tweet refers to a location, and did not expect the participants to find the location on the map. Table 1 shows example tweets and their locations. …

Content truncated to 500 words