Restoring and Mining the Records of the Joseon Dynasty via Neural Language Modeling and Machine Translation

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4031–4042 June 6–11, 2021. ©2021 Association for Computational Linguistics 4031 Restoring and Mining the Records of the Joseon Dynasty via Neural Language Modeling and Machine Translation Kyeongpil Kang Scatter Lab Seoul, South Korea kyeongpil@scatterlab.co.kr Kyohoon Jin Chung-Ang University Seoul, South Korea fhzh123@cau.ac.kr Soyoung Yang KAIST Daejeon, South Korea sy_yang@kaist.ac.kr Soojin Jang Chung-Ang University Seoul, South Korea sujin0110@cau.ac.kr Jaegul Choo KAIST Daejeon, South Korea jchoo@kaist.ac.kr Youngbin Kim Chung-Ang University Seoul, South Korea ybkim85@cau.ac.kr Abstract Understanding voluminous historical records provides clues on the past in various aspects, such as social and political issues and even natural science facts. However, it is generally difficult to fully utilize the historical records, since most of the documents are not written in a modern language and part of the contents are damaged over time. As a result, restoring the damaged or unrecognizable parts as well as translating the records into modern languages are crucial tasks. In response, we present a multi-task learning approach to restore and translate historical documents based on a self- attention mechanism, specifically utilizing two Korean historical records, ones of the most vo- luminous historical records in the world. Ex- perimental results show that our approach sig- nificantly improves the accuracy of the trans- lation task than baselines without multi-task learning. In addition, we present an in-depth exploratory analysis on our translated results via topic modeling, uncovering several signifi- cant historical events. 1 Introduction Historical records are invaluable sources of infor- mation on the lifestyle and scientific records of our ancestors. Humankind has learned how to handle social and political problems by learning from the past. The historical records also serve as the evi- dence of intellectual accomplishment of humanity over time. Given such importance, there has been a great deal of nationwide efforts to preserve these historical records. For instance, UNESCO protects world heritage sites, and experts from all around the world have been converting and restoring historical records in a digital form for long-term preservation. A representative example is the Google Books Li- brary Project1. However, despite the importance of the historical records, it has been challenging to properly utilize the records for the following rea- sons. First, the nontrivial amounts of the documents are partially damaged and unrecognizable due to unfortunate historical events or environments, such as wars and disasters, as well as the weak durability of paper documents. These factors result in difficul- ties to translate and understand the records. Second, as most of the records are written in ancient and out- dated languages, non-experts are difficult to read and understand them. Thus, for their in-depth anal- ysis, it is crucial to recover the damaged parts and properly translate them into modern languages. To address these issues existing in historical records, we formulate them as the task of language modeling, especially for the recovery and neural machine translation, by leveraging the advanced neural networks. Moreover, we apply topic model- …

Content truncated to 500 words