BERT for Monolingual and Cross-Lingual Reverse Dictionary

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 4329–4338 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 4329 BERT for Monolingual and Cross-Lingual Reverse Dictionary Hang Yan, Xiaonan Li, Xipeng Qiu∗, Bocao Deng Shanghai Key Laboratory of Intelligent Information Processing, Fudan University School of Computer Science, Fudan University 2005 Songhu Road, Shanghai, China {hyan19,xnli20,xpqiu}@fudan.edu.cn, dengbocao@gmail.com Abstract Reverse dictionary is the task to find the proper target word given the word description. In this paper, we tried to incorporate BERT into this task. However, since BERT is based on the byte-pair-encoding (BPE) subword en- coding, it is nontrivial to make BERT gen- erate a word given the description. We pro- pose a simple but effective method to make BERT generate the target word for this spe- cific task. Besides, the cross-lingual reverse dictionary is the task to find the proper target word described in another language. Previous models have to keep two different word em- beddings and learn to align these embeddings. Nevertheless, by using the Multilingual BERT (mBERT), we can efficiently conduct the cross- lingual reverse dictionary with one subword embedding, and the alignment between lan- guages is not necessary. More importantly, mBERT can achieve remarkable cross-lingual reverse dictionary performance even without the parallel corpus, which means it can con- duct the cross-lingual reverse dictionary with only corresponding monolingual data. Code is publicly available at https://github.com/ yhcc/BertForRD.git. 1 Introduction Reverse dictionary (Bilac et al., 2004; Hill et al., 2016) is the task to find the proper target word given the word description. Fig. 1 shows an ex- ample of the monolingual and the cross-lingual reverse dictionary. Reverse dictionary should be a useful tool to help writers, translators, and new language learners find a proper word when encoun- tering the tip-of-the-tongue problem (Brown and McNeill, 1966). Moreover, the reverse dictionary can be used for educational evaluation. For ex- ample, teachers can ask the students to describe a ∗Corresponding author. A situation where neither can go forward nor backward. dilemma A French word describes the newly introduced plant species in an area. néophyte Reverse Dictionary Figure 1: An example of the monolingual and cross- lingual reverse dictionary. word, and the correct description should make the reverse dictionary model recall the word. The core of reverse dictionary is to match a word and its description semantically. Early methods (Bi- lac et al., 2004; Shaw et al., 2013) firstly extracted the handcrafted features and then used similarity- based approaches to find the target word. However, since these methods are mainly based on the sur- face form of words, they cannot extract the seman- tic meaning, resulting in bad performance when evaluated on the human-written search query. Re- cent methods usually adopt neural networks to en- code the description and the candidate words into the same semantic embedding space and return the word which is closest to the description (Hill et al., 2016; Zhang et al., 2019). Although current neural methods can extract the semantic representations of the descriptions and words, …

Content truncated to 500 words