Fine-grained Contrastive Learning for Definition Generation

Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1001–1012 November 20–23, 2022. ©2022 Association for Computational Linguistics 1001 Fine-grained Contrastive Learning for Definition Generation Hengyuan Zhang1*, Dawei Li2*, Shiping Yang3, Yanran Li4† 1Shenzhen International Graduate School, Tsinghua University 2Halicio˘glu Data Science Institute, University of California, San Diego 3School of Computer Science, Beijing University of Posts and Telecommunications 4Independent Researcher zhang-hy22@mails.tsinghua.edu.cn, dal034@ucsd.edu, yangshiping@bupt.edu.cn, yanranli.summer@gmail.com Abstract Recently, pre-trained transformer-based mod- els have achieved great success in the task of definition generation (DG). However, previous encoder-decoder models lack effective repre- sentation learning to contain full semantic com- ponents of the given word, which leads to gener- ating under-specific definitions. To address this problem, we propose a novel contrastive learn- ing method, encouraging the model to capture more detailed semantic representations from the definition sequence encoding. According to both automatic and manual evaluation, the ex- perimental results on three mainstream bench- marks demonstrate that the proposed method could generate more specific and high-quality definitions compared with several state-of-the- art models. 1 Introduction When readers find some expressions unfamiliar during reading a text, machines can help. The task of Definition Generation (DG) aims to generate a textual definition for a given word or phrase (the tar- get), according to a surrounding context (the local context) (Ni and Wang, 2017). In addition to assist- ing readers in comprehending expressions, the task of DG is also useful for generating definition when building dictionaries. Recently, pre-trained encoder-decoder models have achieved great successes on this task (Huang et al., 2021; Kong et al., 2022). Despite their successes, the definitions produced by these pre- trained models often contain several types of er- rors (Noraset et al., 2017; Huang et al., 2021). Ac- cording to Table 1, “under-specific problem” is the most frequent error that the generated definition conforms to the general semantics but loses certain parts of meaning of the target word. As presented in Table 2, the definition produced by T5 model is * Equal contribution † Corresponding author Error Types Ratio Under-spcified 9.0% Over-specified 5.5% Self-reference 3.0% Wrong part-of-speech 1.0% Opposite 1.0% Table 1: Ratio of each error type of the definitions gen- erated in Huang et al. (2021). word double Reference twice as great or many Generated characterized by two equal parts or components Table 2: The definition of word “double”, where Refer- ence is from WordNet dictionary and Generated is by T5-Base of Huang et al. (2021). under-specific as it omits the meaning of great in the word “double” under the context “ate a dou- ble portion”. The under-specific problem harms the accuracy of the generated definitions and in turn limits the applications of definition generation techniques in many scenarios. This problem is partially attributed to the de- coder’s inability to fully extract the semantic com- ponents from the word encoding (Li et al., 2020a). For pre-trained encoder-decoder models, they fo- cus on restoring …

Content truncated to 500 words