NLP Analytics in Finance with DoRe: A French 250M Tokens Corpus of Corporate Annual Reports

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 2261–2267 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 2261 NLP Analytics in Finance with DoRe: a French 257M Tokens Corpus of Corporate Annual Reports Corentin Masson∗†, Patrick Paroubek† ∗AMF, †LIMSI-CNRS-U. Paris-Saclay AMF 17 Place de la Bourse, 75002, Paris F LIMSI Bˆat. 507, Rue John Von Neumann, 91400 Orsay F {corentin.masson, pap}@limsi.fr Abstract Recent advances in neural computing and word embeddings for semantic processing open many new applications areas which had been left unaddressed so far because of inadequate language understanding capacity. But this new kind of approaches rely even more on training data to be operational. Corpora for financial applications exists, but most of them concern stock market prediction and are in English. To address this need for the French language and regulation oriented applications which require a deeper understanding of the text content, we hereby present “DoRe”, a French and dialectal French Corpus for NLP analytics in Finance, Regulation and Investment. This corpus is composed of: (a) 2350 Annual Reports from 336 companies among the most capitalized companies in: France (Euronext Paris) & Belgium (Euronext Brussels), covering a time frame from 2009 to 2019, and (b) related MetaData containing information for each company about its ISIN code, capitalization and sector. This corpus is designed to be as modular as possible in order to allow for maximum reuse in different tasks pertaining to Economics, Finance and Regulation. After presenting existing resources, we relate the construction of the DoRe corpus and the rationale behind our choices, concluding on the spectrum of possible uses of this new resource for NLP applications. Keywords: Corpus, French, Finance, Annual Reports 1. Introduction Since the emergence of context insensitive neural word- embeddings (Mikolov et al., 2013) and the revolution of context sensitive pre-trained language models like ELMO (Peters et al., 2018), BERT (Devlin et al., 2019) and XL- NET (Yang et al., 2019), Natural Language Processing (NLP) is gaining more and more popularity as the number of profitable use-cases in various industries identified by both research & private sectors is increasing. Because of the intrinsic language complexity due to its high combina- torics, the reason behind the progress of State-Of-The-Art pretrained models (Shoeybi et al., 2019) is often the avail- ability of more training data and the corresponding com- puting power. Those models are also often trained with ac- cessible general Internet Data as Wikipedia, offering wide vocabularies and topics, allowing NLP tasks to be split into two subtasks: learning the Language Model and then adapt- ing it to the specificities of the target task. The availabil- ity of word embeddings pre-trained on huge amounts of generic (Mikolov et al., 2013; Devlin et al., 2019; Peters et al., 2018; Yang et al., 2019) or specialized (Lee et al., 2019; Beltagy et al., 2019) corpora lowers the barrier-to-entry re- quired to reach a minimal performance for many seman- tic tasks (Devlin et al., 2019). Marketing, Financial, Eco- …

Content truncated to 500 words