Chinese and English Elementary Discourse Units Segmentation based on Bi-LSTM-CRF Model

Chinese and English Elementary Discourse Units Recognition based on Bi-LSTM-CRF Model Yancui Li Chunxiao Lai Jike Feng Hongyu Feng School of Information Engineering, Henan Institute of Science and Technology, Xinxiang , Henan, China Key Laboratory of Advanced Theory and Application in Statistics and Data Science (East China Normal University), Ministry of Education, Shanghai, China Liyancui@hist.edu.cn Abstract Elementary Discourse Unit (EDU) recognition is the basic task of discourse analysis, and the Chinese and English discourse alignment corpus is helpful to the studies of EDU recognition. This paper first builds Chinese-English parallel discourse corpus, in which EDUs are annotated and aligned. Then, we present the framework of Bi-LSTM-CRF EDUs recognition model using word embedding, POS and syntactic features, which can combine the advantage of CRF and Bi- LSTM. The results show that F1 is about 2% higher than the traditional method. Compared with CRF and Bi-LSTM, the Bi-LSTM-CRF model can combine the advantages of them and obtains satisfactory results for Chinese and English EDUs recognition. The experiment of feature contri- bution shows that using all features together can get best result, the syntactic feature outperforms than other features. 1 Introduction Discourse analysis is helpful for the performance of machine translation, question answering, sum- marization and other application. EDU recognition is a basic work in discourse analysis task. Only by recognition EDU, can we make further discourse analysis or other works. At present, the existing Chinese-English parallel corpus only align paragraphs, sentences and other linguistic units, but do not annotate bilingual EDUs alignment, which due to EDU recognition is mainly carried out on monolingual. However, EDUs recognition on Chinese and English is vital to bilingual analysis, machine translation et al. For Example1 is a bilingual sentence of Chinese and English, Chinese EDUs are numbered sequen- tially by e1, e2 and e3, and English EDUs are marked by e1′, e2′ and e3′. Obviously, e1 and e1′, e2 and e2′, e3 and e3′ are alignment pair. Example 1 A) [®É$à�5„u§] e1[üWû_�8§]e2 [n´uˆ"]e3 B) [The Beijing - Hangzhou Grand Canal has been prosperous since ancient times,] e1′ [with both sides of the bank swarming with merchants] e2′ [and well - developed trade.]e3′ The main work of this paper is recognition the EDUs of Chinese and English as much as possible. The following is the contribution of this paper: We annotate Chinese and English discourse alignment corpus, which is first corpus contain EDUs alignment information as far as we know; We get satisfactory results without any handcraft feature by using Bi-LSTM-CRF Model; We conduct to find out the contribution of various model and features. This paper combines existing research and Chinese-English discourse alignment corpus to identify and analyze Chinese-English EDUs. Section 2 builds Chinese-English EDUs alignment corpus; Section 3 describes the Bi-LSTM-CRF model and the framework this paper used; Section 4 reports and analyzes the experimental results; Section 5 overviews the related work; Finally, Section 6 summarizes this paper and points out the future research direction. c⃝2020 China National Conference on Computational Linguistics Published under Creative Commons …

Content truncated to 500 words