Grounding Gradable Adjectives through Crowdsourcing

Grounding Gradable Adjectives through Crowdsourcing Rebecca Sharp, Mithun Paul, Ajay Nagesh, Dane Bell, and Mihai Surdeanu CLU Lab, Dept. of Computer Science, University of Arizona {bsharp, mithunpaul, ajaynagesh, dane, msurdeanu}@email.arizona.edu Abstract In order to build technology that has the ability to answer questions relevant to national and global security, e.g., on food insecurity in certain parts of the world, one has to implement machine reading technology that extracts causal mechanisms from texts. Unfortunately, many of these texts describe these interactions using vague, high-level language. One particular example is the use of gradable adjectives, i.e., adjectives that can take a range of magnitudes such as small or slight. Here we propose a method for estimating specific concrete groundings for a set of such gradable adjectives. We use crowdsourcing to gather human language intuitions about the impact of each adjective, then fit a linear mixed effects model to this data. The resulting model is able to estimate the impact of novel instances of these adjectives found in text. We evaluate our model in terms of its ability to generalize to unseen data and find that it has a predictive R2 of 0.632 in general, and 0.677 on a subset of high-frequency adjectives. Keywords: grounded semantics, gradable adjectives, crowdsourcing 1. Introduction In order to understand the interplay of various entities and events in complex systems such as climate change or crop yields, scientists make use of complex quantitative mod- els (e.g., DSSAT (Jones et al., 2003) or AGMIP (Rosen- zweig et al., 2013)) with qualitative hypotheses (e.g., Parry and Rosenzweig (1993), Zhang el al. (2011), Zhao et al. (2017), inter alia). Typically these models must be hand- generated by domain experts through an expensive process that requires extensive literature review and a large amount of time, resulting in only a small fraction of the available in- formation being processed and incorporated into the mod- els. These models are crucial to predicting the vagaries in such complex systems. A timely and accurate assessment of the factors that affect such systems bears directly on the health of the nation and environment (Elliott et al., 2017; Porwollik et al., 2017). Automated machine reading can help create and validate hypotheses using these models, but there remains a disconnect – often relevant events are writ- ten in high-level language, yet the model requires specific quantities. In particular, when describing events, authors often make use of gradable adjectives, i.e., adjectives (such as small) that can take a range of magnitudes or degrees (e.g., something can be a little small, very small, extraordi- narily small, etc.) When used to describe important changes in model pa- rameters, as with this snippet from a scientific publica- tion: “...doubling of the atmospheric carbon dioxide con- centration will lead to only a small decrease in global crop production.” (Parry and Rosenzweig, 1993), the ability to quantify or ground such adjectives is a critical step for au- tomated machine reading. For example, consider the dif- ference between a small increase in rainfall versus a severe …

Content truncated to 500 words