Assessing Multilinguality of Publicly Accessible Websites

Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022), pages 2108–2116 Marseille, 20-25 June 2022 © European Language Resources Association (ELRA), licensed under CC-BY-NC-4.0 2108 Assessing the Multilinguality of Publicly Accessible Websites Rinalds Vīksna1, 2, Inguna Skadiņa1, 2, Raivis Skadiņš1,2, Andrejs Vasiļjevs1,2, Roberts Rozis1 1Tilde / Vienības gatve 75a, Riga, Latvia 2Faculty of Computing, University of Latvia / Raiņa bulv. 19, Riga, Latvia {name.surname}@tilde.lv Abstract Although information on the Internet can be shared in many languages, the language presence on the World Wide Web is very disproportionate. The problem of multilingualism on the Web, in particular access, availability and quality of information in the world’s languages, has been the subject of UNESCO focus for several decades. Making European websites more multilingual is also one of the focal targets of the Connecting Europe Facility Automated Translation (CEF AT) digital service infrastructure. In order to monitor this goal, alongside other possible solutions, CEF AT needs a methodology and easy to use tool to assess the degree of multilingualism of a given website. In this paper we investigate methods and tools that automatically analyse the language diversity of the Web and propose indicators and methodology on how to measure the multilingualism of European websites. We also introduce a prototype tool based on open-source software that helps to assess multilingualism of the Web and can be independently run at set intervals. We also present initial results obtained with our tool that allows us to conclude that multilingualism on the Web is still a problem not only at the world level, but also at the European and regional level. Keywords: multilingualism, internet, scoring, language equality 1. Introduction The Internet today is multilingual and diverse, it offers many opportunities for sharing information and knowledge. Although the information on the Internet could be shared in all languages that meet certain technical requirements (e.g., established writing system supported in character encoding standards and keyboard layouts), language presence on the World Wide Web is very disproportionate. For example, according to the W3Techs statistics1 about 80% of Web content is in 5 languages: 63.6% in English, 6.9% in Russian, 3.9% in Turkish, 3.6% in Spanish and 3.5% in Persian (Farsi), while all other languages account for only 19% of content. Asymmetries in the volume of online content for different languages can be also observed for Wikipedia2 with 9 languages representing 47.7% of the content (11.1% in English, 10.5% in Cebuano, 4.7% in Swedish, 4.6% German, 4.1% French, 3.6% Dutch, 3.1% Russian, 3% Spanish and Italian), while the remaining 305 languages account for only about 52% of content. Young (2012) in the study about the multilinguality of Wikipedia articles found that 74% of concepts had articles in only one language and 95% of concepts were in fewer than six languages.3 Two recent studies on the multilingual leaders on the Web by CSA Research (Lommel and Sargent, 2019; Sargent and Lommel 2019) examined the language of content by industry. “The Global Website Assessment Index 2019” (Sargent and Lommel 2019) …

Content truncated to 500 words