A Dataset of Syntactic-Ngrams over Time from a Very Large Corpus of English Books

Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 1: Proceedings of the Main Conference and the Shared Task, pages 241–247, Atlanta, Georgia, June 13-14, 2013. c⃝2013 Association for Computational Linguistics A Dataset of Syntactic-Ngrams over Time from a Very Large Corpus of English Books Yoav Goldberg Bar Ilan University∗ yoav.goldberg@gmail.com Jon Orwant Google Inc. orwant@google.com Abstract We created a dataset of syntactic-ngrams (counted dependency-tree fragments) based on a corpus of 3.5 million English books. The dataset includes over 10 billion distinct items covering a wide range of syntactic configura- tions. It also includes temporal information, facilitating new kinds of research into lexical semantics over time. This paper describes the dataset, the syntactic representation, and the kinds of information provided. 1 Introduction The distributional hypothesis of Harris (1954) states that properties of words can be captured based on their contexts. The consequences of this hypoth- esis have been leveraged to a great effect by the NLP community, resulting in algorithms for in- ferring syntactic as well as semantic properties of words (see e.g. (Turney and Pantel, 2010; Baroni and Lenci, 2010) and the references therein). In this paper, we describe a very large dataset of syntactic-ngrams, that is, structures in which the contexts of words are based on their respective po- sition in a syntactic parse tree, and not on their se- quential order in the sentence: the different words in the ngram may be far apart from each other in the sentence, yet close to each other syntactically. See Figure 1 for an example of a syntactic-ngram. The utility of syntactic contexts of words for con- structing vector-space models of word meanings is well established (Lin, 1998; Lin and Pantel, 2001; Pad´o and Lapata, 2007; Baroni and Lenci, 2010). Syntactic relations are successfully used for mod- eling selectional preferences (Erk and Pad´o, 2008; ∗Work performed while at Google. Erk et al., 2010; Ritter et al., 2010; S´eaghdha, 2010), and dependency paths are also used to in- fer binary relations between words (Lin and Pantel, 2001; Wu and Weld, 2010). The use of syntactic- ngrams holds promise also for improving the accu- racy of core NLP tasks such as syntactic language- modeling (Shen et al., 2008) and syntactic-parsing (Chen et al., 2009; Sagae and Gordon, 2009; Co- hen et al., 2012), though most successful attempts to improve syntactic parsing by using counts from large corpora are based on sequential rather than syntactic information (Koo et al., 2008; Bansal and Klein, 2011; Pitler, 2012), we believe this is be- cause large-scale datasets of syntactic counts are not readily available. Unfortunately, most work utiliz- ing counts from large textual corpora does not use a standardized corpora for constructing their models, making it very hard to reproduce results and chal- lenging to compare results across different studies. Our aim in this work is not to present new meth- ods or results, but rather to provide a new kind of a large-scale (based on corpora about 100 times larger than previous efforts) high-quality …

Content truncated to 500 words