Using MT for multilingual covid-19 case load prediction from social media texts

Using MT for multilingual covid-19 case load prediction from social media texts Maja Popovi´c1, Vasudevan Nedumpozhimana2, Meegan Gower3, Sneha Rautmare3, Nishtha Jain3,4, John Kelleher2 ADAPT Centre 1Dublin City University, 2Technological University Dublin, 3Trinity College Dublin (4now at Spoke.ai, Berlin, Germany) name.surname@adaptcentre.ie Abstract In the context of an epidemiological study involving multilingual social media, this paper reports on the ability of machine translation systems to preserve content rel- evant for a document classification task de- signed to determine whether the social me- dia text is related to covid-19. The results indicate that machine translation does pro- vide a feasible basis for scaling epidemio- logical social media surveillance to multi- ple languages. Moreover, a qualitative er- ror analysis revealed that the majority of classification errors are not caused by MT errors. 1 Introduction The work reported in this paper was carried out as part of a covid-19 case load forecasting project. Similar to other work on covid-19 forecasting, e.g. (Rahimi et al., 2021; Wang et al., 2022; Nama- sudra et al., 2023), our baseline system used an auto-regressive approach to case-load prediction. Several studies, however, have pointed to social media as a useful information source for this task (Yousefinaghani et al., 2021; Drinkall et al., 2022). Consequently, we wished to supplement our auto- regressive forecasting with information from so- cial media. Specifically, we used the prevalence of mentions of covid-19 and related concepts in the social media emanating from a location to inform the case load predictions for that location. Given the global nature of covid-19, we wished to make the solution scalable to multiple lan- © 2023 The authors. This article is licensed under a Creative Commons 4.0 licence, no derivative works, attribution, CC- BY-ND. guages. One approach would be to use multilin- gual classifiers. However, having reviewed the literature (see Section 1.1) a decision was made to use machine translation (MT) to translate data sources in other languages into English and then to focus only on developing the text classification and prediction for English. This approach has several technical advantages, such as: (i) many existing NLP resources are designed to work in English (ii) adding new languages involves building a new MT system rather than developing out a new classifica- tion and prediction pipeline for a new language. Interestingly, in this context, as the goal of MT was not translated text as such but enabling down- stream text classification, we did not use any of the usual intrinsic MT evaluation strategies (automatic scores, human evaluation of translation quality cri- teria, error annotation and classification), but ex- trinsic evaluation, namely assessing and analysing the performance of the classifier on the translated English data. Our research questions were: RQ1 How useful is MT for this classification task? In other words: how close is the classi- fication accuracy on translated text to the ac- curacy on original English texts? RQ2 What is the relation between classification and translation errors? In other words: how many of classification errors happened be- cause important terms were …

Content truncated to 500 words