Fine-tuning Neural Language Models for Multidimensional Opinion Mining of English-Maltese Social Data

Proceedings of Recent Advances in Natural Language Processing, pages 309–314 Sep 1–3, 2021. https://doi.org/10.26615/978-954-452-072-4_036 309 Fine-tuning Neural Language Models for Multidimensional Opinion Mining of English-Maltese Social Data Keith Cortis ADAPT Centre, Dublin City University, Glasnevin, Dublin 9, Ireland keith.cortis @adaptcentre.ie Kanishk Verma ADAPT Centre, Dublin City University, Glasnevin, Dublin 9, Ireland kanishk.verma @adaptcentre.ie Brian Davis ADAPT Centre, Dublin City University, Glasnevin, Dublin 9, Ireland brian.davis @adaptcentre.ie Abstract This paper presents multidimensional Social Opinion Mining on user-generated content gathered from newswires and social network- ing services in three different languages: En- glish —a high-resourced language, Maltese —a low-resourced language, and Maltese- English —a code-switched language. Multiple fine-tuned neural classification language mod- els which cater for the i) English, Maltese and Maltese-English languages as well as ii) five different social opinion dimensions, namely subjectivity, sentiment polarity, emotion, irony and sarcasm, are presented. Results per classi- fication model for each social opinion dimen- sion are discussed. 1 Introduction Social Opinion Mining on data obtained from so- cial sources is an evolving research domain tasked with the identification of several opinion dimen- sions, such as subjectivity, sentiment polarity, emo- tion, irony and sarcasm, from noisy user-generated social data spread across heterogeneous sources (Cortis and Davis, 2021b). Currently, Social Opin- ion Mining is used in several real-world scenarios, namely chatbots (Androutsopoulou et al., 2019), adaptive customer online service based on identi- fied customer sentiment and emotion (Yadollahi et al., 2017), tracking of overall customer satisfac- tion for a product or service (Zhao et al., 2019), and detection of changes in customer opinion towards a brand, product or service (Geetha et al., 2017). This paper presents multidimensional Social Opinion Mining on user-generated content gath- ered from newswires and social networking ser- vices in three different languages: English – a high- resourced language, Maltese1 – a low-resourced language, and Maltese-English – a code-switched 1Maltese (Malti) is a Semitic language written in Latin script and is Malta’s national language language. Our aim is use these initial results to improve cross-lingual performance of English- Maltese neural language models. Research applica- tions of the developed classification models include opinion summarisation and fine-grained opinion- ated search of each dimension. This work is in line with Malta’s Strategy and Vision for Artificial Intelligence (Parliamentary Secretariat for Finan- cial Services and Innovation, 2019), with current investment being made in the development of Mal- tese language resources and tools to counter the threat of “digital extinction” for the Maltese lan- guage, which has low technological support avail- able in comparison with other European languages (Rosner et al., 2012). We leverage a novel multidimensional and multilingual social opinion dataset in the socio- economic domain, specifically Malta’s annual Gov- ernment Budget, which comprises social data from the 2018, 2019 and 2020 budgets to fine-tune pre- trained neural language models for benchmarking purposes. 2 Related Work Nguyen et al. (Nguyen et al., 2020) developed the first large-scale pre-trained language model BERTweet for English tweets, which outperforms its baselines. Experiments were conducted on three NLP tasks, namely Part-of-Speech …

Content truncated to 500 words