A Large Automatically-Acquired All-Words List of Multiword Expressions Scored for Compositionality

A Large Automatically-Acquired All-Words List of Multiword Expressions Scored for Compositionality Will Roberts, Markus Egg Humboldt-Universität zu Berlin Unter den Linden 6, 10099 Berlin, Germany {will.roberts, markus.egg}@anglistik.hu-berlin.de Abstract We present and make available a large automatically-acquired all-words list of English multiword expressions scored for compositionality. Intrinsic evaluation against manually-produced gold standards demonstrates that our compositionality estimates are sound, and extrinsic evaluation via incorporation of our list into a machine translation system to better handle idiomatic expressions results in a statistically significant improvement to the system’s BLEU scores. As the method used to produce the list is language-independent, we also make available lists in seven other European languages. Keywords: multiiword expressions, compositionality, machine translation 1. Introduction Multiword expressions (MWEs) are phraseological units, which consist of more than one lexeme and exhibit some kind of idiosyncrasy (Sag et al., 2002); such idiosyncrasy may be lexical (ad hoc), syntactic (by and large), semantic (middle of the road), pragmatic (all aboard), or statistical (black and white but not white and black; these are com- monly known as collocations) (Baldwin and Kim, 2010). In this paper, we present a new linguistic resource, in the form of a large automatically-acquired all-words list of MWEs, which aims to support future research into semanti- cally idiosyncratic MWEs. Semantically idiosyncratic MWEs, or idiomatic expressions, are non-compositional in that their meanings cannot be predicted from their parts; these ex- pressions are used frequently to make language more fluent (Jackendoff, 1997), and often contain word senses not found in other contexts. Thus, identifying non-compositional MWEs presents a clear challenge for fields such as automatic machine translation (MT), information retrieval, natural lan- guage understanding, natural language generation, question answering, text summarisation, and word sense disambigua- tion (McCarthy et al., 2007). In recent years, there has been considerable interest in the MWE community in automati- cally estimating compositionality (Biemann and Giesbrecht, 2011; Reddy et al., 2011; Schulte im Walde et al., 2013; Salehi et al., 2015); however, to the best of our knowledge, this work has hardly been applied to real-world NLP tasks. We set out to distribute a convenient resource representing the best practices gleaned from this work, by automatically scoring the expressions on our list for compositionality. This paper is structured in the following way: Section 2. lists previous work in this area, while section 3. details our acquisition method. Our resource is then evaluated intrinsically against manually-produced gold standards in section 4., and extrinsically, inside a MT system in section 5.. 2. Related work While the resource introduced in this paper is an all-words list acquired automatically, most existing MWE resources are produced manually and focus on a single part of speech (e.g., noun-noun compounds, verb-noun constructions, verb particle constructions, adjective-noun constructions);1 some examples of these are used in Section 4.. Other more general resources include machine-readable dic- tionaries that happen to list MWEs; examples include the TED-MWE bilingual dictionary (Monti et al., 2015), with 2,484 automatically-extracted aligned EN-IT MWEs, and BabelNet (Navigli and Ponzetto, 2012), some of whose 8.5 …

Content truncated to 500 words