Detecting Urgency Status of Crisis Tweets: A Transfer Learning Approach for Low Resource Languages

Proceedings of the 28th International Conference on Computational Linguistics, pages 4693–4703 Barcelona, Spain (Online), December 8-13, 2020 4693 Detecting Urgency Status of Crisis Tweets: A Transfer Learning Approach for Low Resource Languages Efsun Sarioglu Kayi† Mona Diab‡ Linyong Nan†∗ Kathleen McKeown† †Department of Computer Science, Columbia University, New York, USA ‡Facebook AI, Seattle, USA efsun@gwu.edu, linyong.nan@yale.edu, b.qu@columbia.edu, mdiab@fb.com, kathy@cs.columbia.edu Bohan Qu†∗ Abstract We release an urgency dataset that consists of English tweets relating to natural crises. The set is annotated along with annotations of their corresponding urgency status. Additionally, we release evaluation datasets for two low-resource languages, i.e. Sinhala and Odia, and demonstrate an effective zero-shot transfer from English to these two languages by training cross-lingual classi- fiers. We adopt cross-lingual embeddings constructed using different methods to extract features of the tweets, including a few state-of-the-art contextual embeddings such as BERT, RoBERTa and XLM-R. We train a variety of classifier architectures, supervised and semi supervised, on the extracted features. We also further experiment with ensembling the various classifiers. With very limited amounts of labeled data in English and zero data in the low resource languages, we show a successful framework of training monolingual and cross-lingual classifiers using deep learning methods which are known to be data hungry. Specifically, we show that the recent deep contextual embeddings are also helpful when dealing with very small-scale datasets. Classifiers that incorporate RoBERTa yield the best performance for the English urgency detection task, with 25% F1 score absolute improvement over the baselines. For the zero-shot transfer to low re- source languages, classifiers that use LASER features perform the best for Sinhala transfer while XLM-R features benefit the Odia transfer the most. 1 Introduction People all over the world use social media, e.g. Twitter, Facebook, to communicate with the outside world during crises that are either natural or man-made. During an emergent crisis, people post to report their well-being, ask for help, or give updates about the ongoing situation. This type of text data can be utilized to provide situational awareness to support missions such as humanitarian assistance/disaster relief, peacekeeping or infectious disease response. However, with the existence of more than 7,000 languages worldwide, automated human language technology does not exist for many languages.1 A possible solution to this problem is to transfer models learned in high resource language settings such as English to low resource languages (Ruder et al., 2019). In addition, there has been significant research in the use of transfer models in semantic analysis of texts such as sentiment (Socher et al., 2013; Rasooli et al., 2018) and emotion (Tafreshi and Diab, 2018). To this end, we collect and release English, Sinhala and Odia urgency datasets that consist of tweets relating to natural crises, annotated with urgency status. 2 To demonstrate that we are able to effectively transfer the task of urgency detection from English to low-resource languages, we use English annotated tweets for training, and Sinhala/Odia annotated tweets for evaluation only, therefore, exploring zero- shot transfer. Specifically, we consider the following …

Content truncated to 500 words