MultiFin: A Dataset for Multilingual Financial NLP

Findings of the Association for Computational Linguistics: EACL 2023, pages 894–909 May 2-6, 2023 ©2023 Association for Computational Linguistics MULTIFIN: A Dataset for Multilingual Financial NLP Rasmus Kær Jørgensen1,2 Oliver Brandt3 Mareike Hartmann5,6 Xiang Dai4 Christian Igel1 Desmond Elliott1 1Department of Computer Science, University of Copenhagen 2PricewaterhouseCoopers (PwC) 3Independent Researcher 4CSIRO Data61 5Department of Language Science and Technology, Saarland University 6German Research Center for Artificial Intelligence (DFKI) rasmuskj,xiang.dai,igel,de@di.ku.dk obrandt2311@gmail.com mareikeh@lst.de Abstract Financial information is generated and dis- tributed across the world, resulting in a vast amount of domain-specific multilingual data. Multilingual models adapted to the financial do- main would ease deployment when an organiza- tion needs to work with multiple languages on a regular basis. For the development and evalua- tion of such models, there is a need for multilin- gual financial language processing datasets. We describe MULTIFIN– a publicly available finan- cial dataset consisting of real-world article head- lines covering 15 languages across different writ- ing systems and language families. The dataset consists of hierarchical label structure providing two classification tasks: multi-label and multi- class. We develop our annotation schema based on a real-world application and annotate our dataset using both ‘label by native-speaker’ and ‘translate-then-label’ approaches. The evalua- tion of several popular multilingual models, e.g., mBERT, XLM-R, and mT5, show that although decent accuracy can be achieved in high-resource languages, there is substantial room for improve- ment in low-resource languages. 1 Introduction Natural language processing technology has substan- tially improved in recent years due to the general- purpose Transformer model (Vaswani et al., 2017), large-scale self-supervised training from unlabelled corpora (Devlin et al., 2019), and the scaling of both of these to increasingly large datasets and models (Raffel et al., 2020). Nevertheless, there are still benefits to having domain-specific models (Gururan- gan et al., 2020), especially when working with clin- ical (Dai et al., 2022) or financial text (Araci, 2019). The domain of financial text is particularly inter- esting for multilingual NLP, given that it is produced across the world (Lewis et al., 2004; Kær Jørgensen et al., 2021). The text often includes invoices, trans- actions, accounting data, tax policies, and stock mar- ket information, inter-alia, and there is an emerging effort to create monolingual financial BERTs (Fin- BERTs) to process financial text (Araci, 2019; DeS- ola et al., 2019; Yang et al., 2020b; Liu et al., 2021). However, the handling of financial text by multina- tional companies is inherently multilingual, there- fore, there is is a need for datasets to evaluate how well models can process multilingual financial text. To this end, we introduce the MULTIFIN dataset, a publicly available financial dataset consisting of real-world financial article headlines in 15 languages (see examples in Table 1). MULTIFIN is annotated with HIGH-LEVEL and LOW-LEVEL topics for multi- class and multi-label classification, respectively. The dataset is intended as a resource for developing mul- tilingual financial language models. It is the first benchmark for evaluating cross-lingual and multilin- gual performance of financial models across multi- ple languages, writing systems and …

Content truncated to 500 words