NYTWIT: A Dataset of Novel Words in the New York Times

Proceedings of the 28th International Conference on Computational Linguistics, pages 6509–6515 Barcelona, Spain (Online), December 8-13, 2020 6509 NYTWIT: A Dataset of Novel Words in the New York Times Yuval Pinter Georgia Institute of Technology Atlanta, GA, USA uvp@gatech.edu Cassandra L. Jacobs University of Wisconsin Madison, WA, USA cjacobs2@wisc.edu Max Bittker School for Poetic Computation New York, NY, USA maxbittker@gmail.com Abstract We present the New York Times Word Innovation Types dataset, or NYTWIT, a collection of over 2,500 novel English words published in the New York Times between November 2017 and March 2019, manually annotated for their class of novelty (such as lexical derivation, dialectal variation, blending, or compounding). We present baseline results for both uncontextual and contextual prediction of novelty class, showing that there is room for improvement even for state-of-the-art NLP systems. We hope this resource will prove useful for linguists and NLP practitioners by providing a real-world environment of novel word appearance. 1 Introduction Novel words, or Out-Of-Vocabulary words (OOVs), are a pervasive problem in modern natural language processing (Brill, 1995; Young et al., 2018). A common scenario in which this problem appears is that of a pre-trained model containing a word representation component such as an embedding table, encountering a previously-unseen word in a downstream task such as question answering or natural language inference. Multiple lines of work attempt to alleviate the downstream effect of OOV words (M¨uller and Sch¨utze, 2011; Pinter et al., 2017), but most tend to focus on individual categories of OOVs: typographical errors (Sakaguchi et al., 2017), domain-specific terminology (Du et al., 2016), stylistic variability (Eisenstein, 2013; van der Goot, 2019), morphological productivity (Bhatia et al., 2016), or novel named entities (Hoffart et al., 2014). In reality, unseen texts contain all these classes of novelty, and more. OOVs are a typically presented as a significant challenge for generalization or understanding in noisy user-generated text (e.g. Twitter) and/or domain-specific content. Nevertheless, even large corpora that are narrow in domain (edited news stories) contain linguistic innovations, including but not limited to novel morphological processes, typographical errors, and loan words. In this paper, we present a dataset of novel words in English relative to the corpus of articles published by the New York Times (NYT), as collected automatically in real time by a Twitter bot. We name it the New York Times Word Innovation Types corpus, or NYTWIT for short. We annotated each word for one of eighteen linguistically-informed categories of novelty within the context of the NYT corpus, as well as for its date of publication and a retrieval document identifier to enable context extraction.1 To our knowledge, this is the first resource to include novel words along with their contextual information in addition to linguistically-informed annotation, a method that enables expansion beyond dictionary-based methods (Cook and Stevenson, 2010; Dhuliawala et al., 2016; Ahmad, 2000) and decontextualized neol- ogisms (Kulkarni and Wang, 2018). In contrast with resources which provide examples and attestations to lexical forms, NYTWIT was constructed in a corpus-comprehensive manner where novelty …

Content truncated to 500 words