FEWS: Large-Scale, Low-Shot Word Sense Disambiguation with the Dictionary

Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pages 455–465 April 19 - 23, 2021. ©2021 Association for Computational Linguistics 455 FEWS: Large-Scale, Low-Shot Word Sense Disambiguation with the Dictionary Terra Blevins Mandar Joshi Luke Zettlemoyer Paul G. Allen School of Computer Science & Engineering, University of Washington Facebook AI Research, Seattle {blvns, mandar90, lsz}@cs.washington.edu Abstract Current models for Word Sense Disambigua- tion (WSD) struggle to disambiguate rare senses, despite reaching human performance on global WSD metrics. This stems from a lack of data for both modeling and evaluating rare senses in existing WSD datasets. In this paper, we introduce FEWS (Few-shot Exam- ples of Word Senses), a new low-shot WSD dataset automatically extracted from example sentences in Wiktionary. FEWS has high sense coverage across different natural lan- guage domains and provides: (1) a large train- ing set that covers many more senses than pre- vious datasets and (2) a comprehensive eval- uation set containing few- and zero-shot ex- amples of a wide variety of senses. We es- tablish baselines on FEWS with knowledge- based and neural WSD approaches and present transfer learning experiments demonstrating that models additionally trained with FEWS better capture rare senses in existing WSD datasets. Finally, we find humans outperform the best baseline models on FEWS, indicating that FEWS will support significant future work on low-shot WSD. 1 Introduction Word Sense Disambiguation (WSD) is the task of identifying the sense, or meaning, that an ambigu- ous word takes in a specific context. Recent WSD models (Huang et al., 2019; Blevins and Zettle- moyer, 2020; Bevilacqua and Navigli, 2020) have made large gains on the task, surpassing the esti- mated 80% F1 human performance upper bound on WordNet annotated corpora (Navigli, 2009). De- spite this breakthrough, the task remains far from solved: performance on rare and zero-shot senses is still low, and in general, current WSD models struggle to learn senses with few training exam- ples (Kumar et al., 2019; Blevins and Zettlemoyer, C: I liked my friend’s last status... S1: to enjoy... [or] be in favor of. ✓ S2: To show support for, or approval of, something on the Internet by marking it with a vote. C: A transistor-diode matrix is composed of vertical and horizontal wires with a transistor at each intersection. ✓ S1: A grid-like arrangement of electronic components, especially one intended for information coding, decoding or storage. S2: A rectangular arrangement of numbers or terms having various uses [in mathematics]. Figure 1: Sample contexts (C) from FEWS with am- biguous words and a subset of candidate sense defini- tions (S). FEWS covers a wide range of senses, includ- ing new senses and domain-specific senses. 2020). This performance gap stems from limited data for rare senses in current WSD datasets, which are annotated on natural language documents that contain a Zipfian distribution of senses (Postma et al., 2016). More generally, since each word has a different set of candidate senses and new senses are regularly coined, it …

Content truncated to 500 words