Some Chances and Challenges in Applying Language Technologies to Historical Studies in Chinese

Computational Linguistics and Chinese Language Processing Vol. 16, No. 1-2, March/June 2011, pp. 27-46 27 © The Association for Computational Linguistics and Chinese Language Processing Some Chances and Challenges in Applying Language Technologies to Historical Studies in Chinese Chao-Lin Liu∗, Guantao Jin+, Qingfeng Liu#, Wei-Yun Chiu �, and Yih-Soong Yu § Abstract We report applications of language technology to analyzing historical documents in the Database for the Study of Modern Chinese Thoughts and Literature (DSMCTL). We studied two historical issues with the reported techniques: the conceptualization of “huaren” (華人, Chinese people) and the attempt to institute constitutional monarchy in the late Qing dynasty. We also discuss research challenges for supporting sophisticated issues using our experience with DSMCTL, the Database of Government Officials of the Republic of China, and the Dream of the Red Chamber. Advanced techniques and tools for lexical, syntactic, semantic, and pragmatic processing of language information, along with more thorough data collection, are needed to strengthen the collaboration between historians and computer scientists. Keywords: Temporal Analysis, Keyword Trends, Collocation, Chinese Historical Documents, Digital Humanities, Natural Language Processing, Chinese Text Analysis. 1. Introduction Natural language processing (NLP) is a well-known research area in computer science and has been successfully applied to handle and analyze modern textual material in the past decades. ∗ Department of Computer Science, National Chengchi University, Taiwan E-mail: chaolin@nccu.edu.tw + National Chengchi University, Taiwan E-mail: gtqf1908@gmail.com # Institute of Chinese Studies, Chinese University of Hong Kong, Hong Kong E-mail: gtqf1908@gmail.com ‡ Department of Chinese Literature, National Chengchi University, Taiwan § Department of History, National Chengchi University, Taiwan 28 Chao-Lin Liu et al. Whether we can extend the applications of current NLP techniques to historical Chinese text and in the humanistic context (e.g., Xiang & Unsworth, 2006; Hsiang, 2011a; Hsiang, 2011b; Yu, 2012) is a challenge. Word senses and grammar have changed over time, and people have assigned different meanings to the same symbols, phrases, and word patterns. We explored the applications of NLP techniques to support the study of historical issues, based on the textual material from three data sources. These include the Database for the Study of Modern Chinese Thoughts and Literature (DSMCTL),1 the Database of Government Officials of the Republic of China (DGOROC),2 and the Dream of the Red Chamber (DRC).3 DSMCTL is a very large database that contains more than 120 million Chinese characters about Chinese history between 1830 and 1930. DGOROC includes government announcements starting from 1912 to the present. DRC is a famous Chinese novel that was composed in the Qing dynasty. These data sources offer great chances for researchers to study Chinese history and literature, and, due to the huge amount of content, computing technology is expected to provide meaningful help. In this paper, we report how we employed NLP techniques to support historical studies. Chinese text did not contain punctuation until modern days, so we had to face not only the well-known Chinese segmentation problem but also the problem of missing sentence boundaries. In recent attempts, we applied the …

Content truncated to 500 words