Temporal dynamics of semantic relations in word embeddings: an application to predicting armed conflict participants

Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1824–1829 Copenhagen, Denmark, September 7–11, 2017. c⃝2017 Association for Computational Linguistics Temporal dynamics of semantic relations in word embeddings: an application to predicting armed conflict participants Andrey Kutuzov Department of Informatics University of Oslo andreku@ifi.uio.no Erik Velldal Department of Informatics University of Oslo erikve@ifi.uio.no Lilja Øvrelid Department of Informatics University of Oslo liljao@ifi.uio.no Abstract This paper deals with using word embed- ding models to trace the temporal dynam- ics of semantic relations between pairs of words. The set-up is similar to the well-known analogies task, but expanded with a time dimension. To this end, we apply incremental updating of the mod- els with new training texts, including in- cremental vocabulary expansion, coupled with learned transformation matrices that let us map between members of the re- lation. The proposed approach is evalu- ated on the task of predicting insurgent armed groups based on geographical lo- cations. The gold standard data for the time span 1994–2010 is extracted from the UCDP Armed Conflicts dataset. The re- sults show that the method is feasible and outperforms the baselines, but also that important work still remains to be done. 1 Introduction and related work In this research, we make an attempt to model the dynamics of worldwide armed conflicts on the ba- sis of English news texts. To this end, we employ the well-known framework of Continuous Bag-of- Words modeling (Mikolov et al., 2013c) for train- ing word embeddings on the English Gigaword news text corpus (Parker et al., 2011). We learn linear projections from the embeddings of geo- graphical locations where violent armed groups were active to the embeddings of these groups. These projections are then applied to the embed- dings and gold standard data from the subsequent year, thus predicting what entities act as violent groups in the next time slice. To evaluate our approach, we adapt the UCDP Armed Conflict Dataset (Gleditsch et al., 2002; Allansson et al., 2017) (see Section 2 for details). Here is a simplified example of the task: given that in 2003, the Kashmir Liberation Front and ULFA were involved in armed conflicts in India, and Lord’s Resistance Army in Uganda, predict en- tities playing the same role in 2004 in Iraq (the cor- rect answers are Ansar al-Islam, al-Mahdi Army and Islamic State). The nature of the task is con- ceptually similar to that of analogical reasoning, but with the added complexity of temporal change. Attempts to detect semantic change using un- supervised methods have a long history. Signif- icant results have already been achieved in em- ploying word embeddings to study diachronic lan- guage change. Among others, Eger and Mehler (2016) show that the embedding of a given word for a given time period to a large extent is a lin- ear combination of its embeddings for the pre- vious time periods. Hamilton et al. (2016) pro- posed an important distinction between cultural shifts and linguistic drifts. They proved that global embedding-based …

Content truncated to 500 words