Scalar Adjective Identification and Multilingual Ranking

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4653–4660 June 6–11, 2021. ©2021 Association for Computational Linguistics 4653 Scalar Adjective Identification and Multilingual Ranking Aina Garí Soler Université Paris-Saclay CNRS, LISN 91400, Orsay, France aina.gari@limsi.fr Marianna Apidianaki Department of Digital Humanities University of Helsinki Helsinki, Finland marianna.apidianaki@helsinki.fi Abstract The intensity relationship that holds between scalar adjectives (e.g., nice < great < wonder- ful) is highly relevant for natural language in- ference and common-sense reasoning. Previ- ous research on scalar adjective ranking has focused on English, mainly due to the avail- ability of datasets for evaluation. We intro- duce a new multilingual dataset in order to pro- mote research on scalar adjectives in new lan- guages. We perform a series of experiments and set performance baselines on this dataset, using monolingual and multilingual contextual language models. Additionally, we introduce a new binary classification task for English scalar adjective identification which examines the models’ ability to distinguish scalar from relational adjectives. We probe contextualised representations and report baseline results for future comparison on this task. 1 Introduction Scalar adjectives relate the entities they modify to specific positions on the evoked scale (e.g., GOOD- NESS, TEMPERATURE, SIZE): A wonderful view is nicer than a good view, and one would proba- bly prefer a delicious to a tasty meal. But not all adjectives express intensity or degree. Relational adjectives are derived from nouns (e.g., wood → wooden, chemistry → chemical), have no antonyms and serve to classify nouns (e.g., a wooden table, a chemical substance) (McNally and Boleda, 2004). The distinction between scalar and relational ad- jectives is an important one. Identifying adjectives that express intensity can serve to assess the emo- tional tone of a given text, as opposed to words that mostly contribute to its descriptive content. Addi- tionally, estimating the intensity of a scalar adjec- tive is useful for textual entailment (wonderful |= good but good ̸|= wonderful), product review anal- ysis and recommendation systems, emotional chat- bots and question answering (de Marneffe et al., 2010). DEMELO EN dim < gloomy < dark < black FR terne < sombre < foncé < noir ES sombrío < tenebroso < oscuro < negro EL αμυδρός || αχνός < μουντός < σκοτεινός< μαύρος WILKINSON EN bad < awful < terrible < horrible FR mauvais < affreux < terrible < horrible ES malo < terrible < horrible < horroroso EL κακός < απαίσιος < τρομερός < φρικτός Table 1: Example translations from each dataset. “||” indicates adjectives at the same intensity level (ties). Work on scalar adjectives has until now evolved around pre-compiled datasets (de Melo and Bansal, 2013; Taboada et al., 2011; Wilkinson and Oates, 2016; Cocos et al., 2018). Reliance on external resources has also restricted research to English, and has led to the prevalence of pattern-based and lexicon-based approaches. Recently, Garí Soler and Apidianaki (2020) showed that BERT represen- tations (Devlin et al., 2019) encode intensity rela- tionships between English …

Content truncated to 500 words