A BERT-based Dual Embedding Model for Chinese Idiom Prediction

Proceedings of the 28th International Conference on Computational Linguistics, pages 1312–1322 Barcelona, Spain (Online), December 8-13, 2020 1312 A BERT-based Dual Embedding Model for Chinese Idiom Prediction Minghuan Tan School of Information Systems Singapore Management University mhtan.2017@phdcs.smu.edu.sg Jing Jiang School of Information Systems Singapore Management University jingjiang@smu.edu.sg Abstract Chinese idioms are special fixed phrases usually derived from ancient stories, whose meanings are oftentimes highly idiomatic and non-compositional. The Chinese idiom prediction task is to select the correct idiom from a set of candidate idioms given a context with a blank. We pro- pose a BERT-based dual embedding model to encode the contextual words as well as to learn dual embeddings of the idioms. Specifically, we first match the embedding of each candidate idiom with the hidden representation corresponding to the blank in the context. We then match the embedding of each candidate idiom with the hidden representations of all the tokens in the context thorough context pooling. We further propose to use two separate idiom embeddings for the two kinds of matching. Experiments on a recently released Chinese idiom cloze test dataset show that our proposed method performs better than the existing state of the art. Ablation exper- iments also show that both context pooling and dual embedding contribute to the improvement of performance. 1 Introduction In this paper, we study Chinese idiom prediction, a language understanding problem that has not been extensively explored before in computational linguistics. Chinese idioms, mainly Chengyu (成语) (set phrases) (Wang and Yu, 2010; Wang, 2019), have fixed forms in structure; the component charac- ters (mostly four) cannot be changed. Chinese idioms are characterized by rich contents, concise forms and frequent use (Wang, 2019) with properties of structural regularity, semantic fusion, and functional integrity (Shao, 2018; Wang, 2019). Chinese idioms are commonly used in both written and spoken Chinese, and understanding Chinese idioms is important for learning Chinese as a second language. The meaning of each Chinese idiom may not be literally understood through the composition of its characters, especially for those which are derived from historical stories or formulated using ancient Chinese grammars. For example, “一定不易” is literally interpreted as “it must be not easy” in modern Chinese. However, the idiom is constructed from grammars and word senses of ancient Chinese. Its idiomatic meaning is “once decided, never change”, which is not even close to the literal meaning. As a result, the usage of Chinese idioms poses a challenge on language understanding not only for humans but also for artificial intelligence. Due to their pervasive usage, Chinese idiom prediction is an important task in Chinese language understanding. There have been several studies focusing on representing Chinese idioms using neural network mod- els (Jiang et al., 2018; Liu et al., 2019b), but they were limited by the amount of data available for training. Recently, Zheng et al. (2019) released a large-scale Chinese IDiom Dataset (ChID) to facilitate machine comprehension of Chinese idioms. The ChID dataset contains more than 500K passages and 600K blanks, making it …

Content truncated to 500 words