Toward Algorithmic Discovery of Biographical Information in Local Gazetteers of Ancient China

Toward Algorithmic Discovery of Biographical Information in Local Gazetteers of Ancient China Chao-Lin Liu† Chih-Kai Huang§ Hongsu Wang‡ Peter K. Bol! †§Department of Computer Science, National Chengchi University, Taiwan ‡!Institute for Quantitative Social Science, Harvard University, USA †Graduate Institute of Linguistics, National Chengchi University, Taiwan {†chaolin,§102753029}@nccu.edu.tw, {‡hongsuwang, !pkbol}@fas.harvard.edu Abstract1 Difangzhi (地方志) is a large collection of local gazetteers complied by local govern- ments of China, and the documents provide invaluable information about the host locali- ty. This paper reports the current status of using natural language processing and text mining methods to identify biographical in- formation of government officers so that we can add the information into the China Bio- graphical Database (CBDB), which is hosted by Harvard University. Information offered by CBDB is instrumental for human histori- ans, and serves as a core foundation for au- tomatic tagging systems, like MARKUS of the Leiden University. Mining texts in Difangzhi is not easy partially because there is litter knowledge about the grammars of literary Chinese so far. We employed tech- niques of language modeling and conditional random fields to find person and location names and their relationships. The methods were evaluated with realistic Difangzhi data of more than 2 million Chinese characters written in literary Chinese. Experimental re- sults indicate that useful information was discovered from the current dataset. 1 Introduction Person and location names are two crucial in- gredients for studying historical documents. Knowing the participants and locations provides a solid foundation for detecting and reasoning about the developments of historical events. Detecting temporal markers is also very im- portant for historical studies, yet, for Chinese history, it is relatively easier to spot the temporal 1 An extended version of this paper appears in the proceedings of the Third Big Humanities Data Workshop in the 2015 IEEE Int’l Conf. on Big Data (Liu et al., 2015). The main contents of this paper and the workshop paper are the same, while the workshop paper is an extended version. markers because the names of the dynasties and reign periods (年號, nian2 hao4) are known and stable. We apply techniques of natural language processing and machine learning to find person names, location names, and their relationships in Difangzhi (地方志, DFZ henceforth) in the present work, aiming to enrich the contents of the China Biographical Database (Bol, 2012). DFZ is a general name for a large number of local gazetteers that were compiled by local governments of different levels in China since as early as the 6th century AD (cf. Hargett, 1996). DFZ contain a wide range of information about their host locations, and the biographical information about the government officers is our current focus. The main barrier for achieving our goals is that there is little completed work in the litera- ture about the grammars for literary Chinese, while grammars are central for extracting named entities like person and location names from texts with computational methods (Gao et al., 2005; Nadeau and Sekine, 2007). Figure 1 shows the image of a sample …

Content truncated to 500 words