Continuous Representation of Location for Geolocation and Lexical Dialectology using Mixture Density Networks

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 167–176 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Continuous Representation of Location for Geolocation and Lexical Dialectology using Mixture Density Networks Afshin Rahimi Timothy Baldwin Trevor Cohn School of Computing and Information Systems The University of Melbourne arahimi@student.unimelb.edu.au {tbaldwin,t.cohn}@unimelb.edu.au Abstract We propose a method for embedding two- dimensional locations in a continuous vec- tor space using a neural network-based model incorporating mixtures of Gaussian distributions, presenting two model vari- ants for text-based geolocation and lexi- cal dialectology. Evaluated over Twitter data, the proposed model outperforms con- ventional regression-based geolocation and provides a better estimate of uncertainty. We also show the effectiveness of the rep- resentation for predicting words from loca- tion in lexical dialectology, and evaluate it using the DARE dataset. 1 Introduction Geolocation is an essential component of appli- cations such as traffic monitoring (Emadi et al., 2017), human mobility pattern analysis (McNeill et al., 2016; Dredze et al., 2016) and disaster re- sponse (Ashktorab et al., 2014; Wakamiya et al., 2016), as well as targeted advertising (Anagnos- topoulos et al., 2016) and local recommender sys- tems (Ho et al., 2012). Although Twitter provides users with the means to geotag their messages, less than 1% of users opt to turn on geotagging, so third- party service providers tend to use profile data, text content and network information to infer the lo- cation of users. Text content is the most widely used source of geolocation information, due to its prevalence across social media services. Text-based geolocation systems use the geo- graphical bias of language to infer the location of a user or message using models trained on geotagged posts. The models often use a representation of text (e.g. based on a bag-of-words, convolutional or recurrent model) to predict the location either in real-valued latitude/longitude coordinate space or in discretised region-based space, using regression or classification, respectively. Regression models, as a consequence of minimising squared loss for a unimodal distribution, predict inputs with multiple targets to lie between the targets (e.g. a user who mentions content in both NYC and LA is predicted to be in the centre of the U.S.). Classification mod- els, while eliminating this problem by predicting a more granular target, don’t provide fine-grained predictions (e.g. specific locations in NYC), and also require heuristic discretisation of locations into regions (e.g. using clustering). Mixture Density Networks (“MDNs”: Bishop (1994)) alleviate these problems by representing location as a mixture of Gaussian distributions. Given a text input, an MDN can generate a mixture model in the form of a probability distribution over all location points. In the example of a user who talks about both NYC and LA, e.g., the model will predict a strong Gaussian component in NYC and another one in LA, and also provide an estimate of uncertainty over all the coordinate space. Although MDNs are not new, they have not found widespread use in inverse regression prob- lems where for a …

Content truncated to 500 words