Combining Discourse Markers and Cross-lingual Embeddings for Synonym–Antonym Classification

Proceedings of NAACL-HLT 2019, pages 3899–3905 Minneapolis, Minnesota, June 2 - June 7, 2019. c⃝2019 Association for Computational Linguistics 3899 Combining Discourse Markers and Cross-lingual Embeddings for Synonym–Antonym Classification Michael Roth1, Shyam Upadhyay2 1Institute for Natural Language Processing, University of Stuttgart 2Department of Computer and Information Science, University of Pennsylvania rothml@ims.uni-stuttgart.de, shyamupa@seas.upenn.edu Abstract It is well-known that distributional seman- tic approaches have difficulty in distinguish- ing between synonyms and antonyms (Grefen- stette, 1992; Padó and Lapata, 2003). Recent work has shown that supervision available in English for this task (e.g., lexical resources) can be transferred to other languages via cross- lingual word embeddings. However, this kind of transfer misses monolingual distributional information available in a target language, such as contrast relations that are indicative of antonymy (e.g., hot…while…cold). In this work, we improve the transfer by exploit- ing monolingual information, expressed in the form of co-occurrences with discourse markers that convey contrast. Our approach makes use of less than a dozen markers, which can eas- ily be obtained for many languages. Compared to a baseline using only cross-lingual embed- dings, we show absolute improvements of 4– 10% F1-score in Vietnamese and Hindi. 1 Introduction Recent work has shown that monolingual word embeddings in different languages can be aligned in an unsupervised manner (Artetxe et al., 2018; Kementchedjhieva et al., 2018). The resulting cross-lingual embeddings can be used to share supervision for lexical classification tasks across languages, when annotated data is not available in one language. For instance, a model for dis- tinguishing lexical relations such as hypernymy and meronymy can be transferred to other lan- guages (Glavaš and Vulić, 2018). However, this kind of transfer, using only cross-lingual em- beddings, misses useful monolingual information available in the target language. In this paper, we consider one lexical clas- sification task, namely the distinction between synonyms and antonyms, which is important English German hot cold new old ruhig kalt kühl hitzig scharf heiß kalt …WOBEI …heiß (‘cold …WHILE …hot’) co-occurrences with discourse markers Figure 1: Supervision for distinguishing antonyms from synonyms can be derived using discourse mark- ers. Here, antonyms available in English (denoted by solid edge between hot and cold) are translated to German via cross-lingual word embeddings. Us- ing co-occurrences with discourse markers indicative of antonymy (shown in box), we can identify pairs of words in the n-best translations (clouds) that are also antonymous (e.g., dashed edge between kalt and heiß). for downstream applications such as contradic- tion detection (Harabagiu et al., 2006; Marneffe et al., 2008; Voorhees, 2008) and machine trans- lation (Marton et al., 2011). To facilitate better transfer, we propose to use monolingual informa- tion in the form of word co-occurrences in con- trast relations, in addition to cross-lingual embed- dings (see Figure 1). In particular, we utilize the fact that discourse markers conveying contrast are more likely to be surrounded by antonyms than synonyms (e.g., hot…while…cold), as shown by Roth and Schulte im Walde (2014). Our analysis reveals that (1) fine-grained se- mantic information that is …

Content truncated to 500 words