EMBEDDIA project: Cross-Lingual Embeddings for Less- Represented Languages in European News Media

EMBEDDIA project: Cross-Lingual Embeddings for Less- Represented Languages in European News Media Senja Pollak and Andraˇz Pelicon Jozef Stefan Institute Jamova 39, Ljubljana, Slovenia senja.pollak,andraz.pelicon@ijs.si Abstract The EMBEDDIA project developed a range of resources and methods for less- resourced EU languages, focusing on ap- plications for media industry, including keyword extraction, comment moderation and article generation. 1 Introduction In the EU, websites and online services for citi- zens offer resources in national local languages, and often only provide a second language (usu- ally English) when absolutely needed. For the EU to realise a truly equitable, open, multilingual on- line content and tools to support its management, new multilingual technologies which do not rely on translation of text between languages are ur- gently needed. The aim of EMBEDDIA was to address these challenges by leveraging innovations in the use of cross-lingual and multilingual embed- dings coupled with deep neural networks to allow existing monolingual resources to be used across languages, leveraging their high speed of opera- tion for near real-time applications, without the need for large computational resources. Across more than three years (01/01/2019 to 31/12/2021), six academic partners (Jozef Stefan Institute, the coordinating partner, Queen Mary University of London, University of Ljubljana, University of La Rochelle, University of Helsinki and University of Edinburgh) and four industry partners (TEXTA O ¨U, As Ekspress Meedia, Finnish News Agency STT and Trikoder d.o.o.) developed novel solu- tions with focus on less-represented EU languages, and tested them in real-world media production contexts. The main scientific goals of the project were to: © 2022 The authors. This article is licensed under a Creative Commons 3.0 licence, no derivative works, attribution, CC- BY-ND. • Develop the embeddings technology for new generation NLP tools, which are both multi- lingual (able to deal with multiple languages) and cross-lingual (transfer easily across lan- guages). • Develop tools and resources for less- resourced morphologically rich EU lan- guages, including Croatian, Estonian, Finnish, Latvian, Lithuanian and Slovenian. • Leverage tools for well-resourced languages to be used for less-represented languages. The project was strongly committed to address the challenges in news media industry, including: • Comment analysis with mono- and cross- lingual applications in offensive speech filter- ing, fake news spreaders detection and senti- ment analysis. • News analysis with applications for key- word extraction, named entity recognition, news sentiment detection, viewpoints analy- sis, topic modelling, news linking, etc. • News generation including text generation from structured data and headline generation. 1.1 Acknowledgements This work has been supported by the EU’s Horizon 2020 RIA under grant 825153 (EMBEDDIA), as well as ARRS core programme P2-0103. 2 Selected outputs 2.1 Datasets EMBEDDIA has publicly released news and com- ments datasets (Pollak et al., 2021) in Estonian, Croatian, Russian and Latvian under the CC BY- NC-ND 4.0 license. We also created a set of novel benchmarks for evaluation, including CoSimLex dataset of word similarity in context (Armendariz et al., 2020) and cross-lingual analogy datasets (Ulˇcar et al., 2020). 2.2 Pretrained embeddings Several monolingual and …

Content truncated to 500 words