WikiHan: A New Comparative Dataset for Chinese Languages

Proceedings of the 29th International Conference on Computational Linguistics, pages 3563–3569 October 12–17, 2022. 3563 WikiHan: A New Comparative Dataset for Chinese Languages Kalvin Chang and Chenxuan Cui and Youngmin Kim and David Mortensen Language Technologies Institute, Carnegie Mellon University {kalvinc, cxcui, youngmik, dmortens}@cs.cmu.edu Abstract Most comparative datasets of Chinese varieties are not digital; however, Wiktionary includes a wealth of transcriptions of words from these varieties. The usefulness of these data is lim- ited by the fact that they use a wide range of variety-specific romanizations, making data difficult to compare. The current work collects this data into a single constituent (IPA, or In- ternational Phonetic Alphabet) and structured form (TSV) for use in comparative linguistics and Chinese NLP. At the time of writing, the dataset contains 67,943 entries across 8 vari- eties and Middle Chinese.1 The dataset is val- idated on a protoform reconstruction task us- ing an encoder-decoder cross-attention archi- tecture (Meloni et al., 2021), achieving an ac- curacy of 54.11%, a PER (phoneme error rate) of 17.69%, and a FER (feature error rate) of 6.60%. 1 Introduction The Chinese language family consists of many mutually unintelligible languages, which linguists cluster into seven well-established subgroups (Mandarin, Yue, Wu, Min, Hakka, Gan, and Xi- ang) and a few more debated subgroups (Jin, Hui, and Pinghua; Handel 2015). Each subgroup is made of varieties that may or may not be mutually unintelligible. Most scholars of Sinitic believe that all varieties except for Min languages descended from Middle Chinese (Handel, 2015), which is a set of old varieties documented in Qièyùn (切韻), a rhyme dictionary compiled in 601 CE. Today, there are a total of 1.3 billion speakers of Sinitic varieties, making the family one of the largest in terms of speaker count (Eberhard et al., 2022). Eight of the subgroups have at least tens of 1The data is available at https://github. com/cmu-llab/wikihan and the code is avail- able at https://github.com/cmu-llab/ meloni-2021-reimplementation/. millions of speakers. In all Chinese-speaking soci- eties except for Hong Kong, Mandarin is the lan- guage of administration, meaning that other vari- eties are not commonly written, although it is possi- ble to do so, either with Han characters (Chinese lo- gographic characters) or romanized orthographies. Because Standard Written Chinese diverged from the spoken varieties over time (Chen, 2015), it is unclear which Han characters should be used for transcription, for many words in the spoken va- rieties. However, scholars can trace from which “original character” in Middle Chinese or Old Chi- nese the modern word descended (běnzì 本字) (Yang, 2000). To determine the pronunciations of these orig- inal characters in Middle Chinese, historical lin- guists and dialectologists compare the pronunci- ations of words across modern Sinitic varieties. More broadly, protoform reconstruction is the in- ference of morphemes or words as they appeared in the ancestral languages of a set of daughter languages. Cognates—words deemed to descend from the same ancestral form—are inputs to the re- construction, while this ancestral form is the output of the reconstruction. See …

Content truncated to 500 words