UniMelb at SemEval-2019 Task 12: Multi-model combination for toponym resolution

Proceedings of the 13th International Workshop on Semantic Evaluation (SemEval-2019), pages 1313–1318 Minneapolis, Minnesota, USA, June 6–7, 2019. ©2019 Association for Computational Linguistics 1313 UniMelb at SemEval-2019 Task 12: Multi-model Combination for Toponym Resolution Haonan Li♠ Minghan Wang♠ Maria Vasardani♥ Martin Tomko♥ Timothy Baldwin♠ ♠ School of Computing and Information Systems ♥ Department of Infrastructure Engineering The University of Melbourne {haonanl5,minghanw}@student.unimelb.edu.au, {maria.vasardani, tomkom}@unimelb.edu.au, tb@ldwin.net Abstract This paper describes our submission SemEval-2019 Task 12 on toponym resolutionto in scientific papers. We train separate NER models for toponym detection over text ex- tracted from tables vs. text from the body of the paper, and train another auxiliary model to eliminate mis-detected toponyms. For to- ponym disambiguation, we use an SVM clas- sifier with hand-engineered features. Our best model achieved a strict micro-F1 score of 80.92% and overlap micro-F1 score of 86.88% in the toponym detection subtask, ranking 2nd out of 8 teams on F1 score. For toponym dis- ambiguation and end-to-end resolution, we of- ficially ranked 2nd and 3rd, respectively. 1 Introduction Toponym resolution (TR) refers to the task of automatically assigning geographic references to place names in text, which has applications in question answering and information retrieval tasks (Leidner, 2008; Daoud and Huang, 2013; Vasar- dani et al., 2013), user geolocation prediction (Roller et al., 2012; Han et al., 2014; Rahimi et al., 2015), and historical research (Grover et al., 2010). This paper describes our system entry to the To- ponym resolution in scientific paper task of Se- mEval 2019 (Weissenbacher et al., 2019). The task consists of three subtasks: toponym detection, to- ponym disambiguation, and end-to-end toponym resolution. For the toponym detection task, we extract ta- bles from the full text and train separate BiLSTM- ATTN models for each. For tables, the model cap- tures the horizontal row-wise structure of the ta- ble. For non-table content, the model can capture syntactic and semantic features. In both cases, we use a deep contextualized word representation — ELMo (Peters et al., 2018) — to represent each token. After detecting toponyms, we use an orga- nization name detection model to eliminate mis- detected toponyms that are actually part of an or- ganization name. For the toponym disambiguation task, we first construct a candidate set by search- ing toponyms on GeoNames.1 Then, we manu- ally construct features based on search results, and finally, train an SVM model to disambiguate the locations. For the end-to-end resolution task, we pipeline the two aforementioned steps. Our work makes the following contributions: • we show that training separate models for ta- ble and non-table portions of the paper is bet- ter than simply training one model over the full text; • we show that contextualized word represen- tation boosts performance; • we show that auxiliary organization name recognition model is helpful for toponym detection, and better than training a single named entity recognizer (NER). 2 Toponym Detection Figure 1 shows our workflow on the toponym de- tection task, which consist of 4 parts: (a) pre- processing, which contains …

Content truncated to 500 words