CLDFBench: Give Your Cross-Linguistic Data a Lift

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 6995–7002 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 6995 CLDFBench: Give Your Cross-Linguistic Data a Lift Robert Forkel1 and Johann-Mattis List1 1Department of Linguistic and Cultural Evolution, Max Planck Institute for the Science of Human History, Jena {forkel, list}@shh.mpg.de Abstract While the amount of cross-linguistic data is constantly increasing, most datasets produced today and in the past cannot be considered FAIR (findable, accessible, interoperable, and reproducible). To remedy this and to increase the comparability of cross-linguistic re- sources, it is not enough to set up standards and best practices for data to be collected in the future. We also need consistent workflows for the “retro-standardization” of data that has been published during the past decades and centuries. With the Cross-Linguistic Data For- mats initiative, first standards for cross-linguistic data have been presented and successfully tested. So far, however, CLDF creation was hampered by the fact that it required a considerable degree of computational proficiency. With cldfbench, we introduce a framework for the retro-standardization of legacy data and the curation of new datasets that drastically simplifies the creation of CLDF by providing a consistent, reproducible workflow that rigorously supports version control and long term archiving of research data and code. The framework is distributed in form of a Python package along with usage information and examples for best practice. This study introduces the new framework and illustrates how it can be applied by showing how a resource containing structural and lexical data for Sinitic languages can be efficiently retro-standardized and analyzed. Keywords: cross-linguistic data, retro-standardization, data curation 1. Introduction While the amount of cross-linguistic data is constantly in- creasing, this “data deluge” has the potentially detrimental effect of making it easier to neglect “old” data – since sim- ilar enough “new” data may be available. For many low- resource languages – in particular those ones which are on the brink of extinction – this means they will effectively disappear from research when existing data is of low qual- ity and new data can no longer be obtained. Thus, mak- ing existing data ready for re-use – “lifting” it into current research environments – is vital for any cross-linguistic re- search. At the same time, it is also vital to make sure that new data that is being constantly produced by the linguistic community is provided in such a form that it can easily be lifted to overarching standards. While initial efforts to increase the reproducibility of re- search in linguistics have been undertaken (Berez-Kroeker et al., 2018) and large datasets which aggregate linguistic resources in order to make them apt for cross-linguistic investigations have been increasingly prepared during the past two decades (Dryer and Haspelmath, 2011; Haspel- math and Tadmor, 2009; Key and Comrie, 2016), we still face a situation in which the majority of linguistic data does not conform to the principles of FAIR data in the sense of Wilkinson et …

Content truncated to 500 words