A Framework for Shared Agreement of Language Tags beyond ISO 639

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 3333–3339 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 3333 A Framework for Shared Agreement of Language Tags beyond ISO 639 Frances Gillis-Webber, Sabine Tittel Department of Computer Science, University of Cape Town, Heidelberg Academy of Sciences and Humanities Cape Town, South Africa, Heidelberg, Germany fran@fynbosch.com, sabine.tittel@urz.uni-heidelberg.de Abstract The identification and annotation of languages in an unambiguous and standardized way is essential for the description of linguistic data. It is the prerequisite for machine-based interpretation, aggregation, and re-use of the data with respect to different languages. This makes it a key aspect especially for Linked Data and the multilingual Semantic Web. The standard for language tags is defined by IETF’s BCP 47 and ISO 639 provides the language codes that are the tags’ main constituents. However, for the identification of lesser-known languages, endangered languages, regional varieties or historical stages of a language, the ISO 639 codes are insufficient. Also, the optional language sub-tags compliant with BCP 47 do not offer a possibility fine-grained enough to represent linguistic variation. We propose a versatile pattern that extends the BCP 47 sub-tag privateuse and is, thus, able to overcome the limits of BCP 47 and ISO 639. Sufficient coverage of the pattern is demonstrated with the use case of linguistic Linked Data of the endangered Gascon language. We show how to use a URI shortcode for the extended sub-tag, making the length compliant with BCP 47. We achieve this with a web application and API developed to encode and decode the language tag. Keywords: language tags, language codes, Linked Data 1. Introduction Data modeling and the processing of linguistic resources requires the use of language tags. Language tags serve to identify the language of a resource and the linguistic en- tities therein. In particular, language tagging is a key as- pect when modeling linguistic resources in Resource De- scription Framework (RDF) (Manola and Miller, 2014) fol- lowing the principles of Linked Data (LD) (Berners-Lee, 2006). Language tags are typically compiled of language codes, with additional sub-tags where applicable. By adhering to established standards, language tags support a machine- based, cross-resource interoperability with respect to the identification of the language of given data. BCP 47 (Best Current Practice, by the Internet Engineering Task Force (IETF)) defines the standard for language tags (Phillips and Davis, 2009). A language code, in the form of a two- or three-character identification such as ‘en’ for English, is the main constituent of a language tag and is provided by the ISO 639 standard (International Organization for Standard- ization, nd). The codes must be unambiguous to be able to disambiguate the case when one language name refers to several languages, or one language is denoted by sev- eral names. However, for the identification and annotation of lesser-known languages, endangered languages, regional varieties or historical stages of a language, the ISO 639 codes are insufficient. Also, the optional extended language sub-tags compliant with …

Content truncated to 500 words