Crosslingual and Multilingual Construction of Syntax-Based Vector Space Models

Crosslingual and Multilingual Construction of Syntax-Based Vector Space Models Jason Utt and Sebastian Padó Institut für Maschinelle Sprachverarbeitung Universität Stuttgart [uttjn|pado]@ims.uni-stuttgart.de Abstract Syntax-based distributional models of lexical semantics provide a flexible and linguistically adequate representation of co-occurrence infor- mation. However, their construction requires large, accurately parsed corpora, which are un- available for most languages. In this paper, we develop a number of meth- ods to overcome this obstacle. We describe (a) a crosslingual approach that constructs a syntax-based model for a new language requir- ing only an English resource and a translation lexicon; and (b) multilingual approaches that combine crosslingual with monolingual infor- mation, subject to availability. We evaluate on two lexical semantic benchmarks in Ger- man and Croatian. We find that the models exhibit complementary profiles: crosslingual models yield higher accuracies while monolin- gual models provide better coverage. In addi- tion, we show that simple multilingual models can successfully combine their strengths. 1 Introduction Building on the Distributional Hypothesis (Harris, 1954; Miller and Charles, 1991), which states that words occurring in similar contexts are similar in meaning, distributional semantic models (DSMs) rep- resent a word’s meaning via its occurrence in context in large corpora. Vector spaces, the most widely used type of DSMs, represent words as vectors in a high- dimensional space whose dimensions correspond to features of the words’ contexts. Word spaces repre- sent the simplest case of DSMs in which the dimen- sions are simply the context words (Schütze, 1992). A notable subclass of DSMs are syntax-based mod- els (Lin, 1998; Baroni and Lenci, 2010) which use (lexicalized) syntactic relations as dimensions. They are able to model more fine-grained distinctions than word spaces and have been found to be useful for tasks such as selectional preference learning (Erk et al., 2010), verb class induction (Schulte im Walde, 2006), analogical reasoning (Turney, 2006), and alter- nation discovery (Joanis et al., 2006). Despite their flexibility and usefulness, syntax-based DSMs are used less often than word-based spaces. An impor- tant reason is that their construction requires accurate parsers, which are unavailable for many languages. In addition, syntax-based DSMs are inherently more sparse than word spaces, which calls for a large cor- pus of well parsable data. It is thus not surprising that besides English (Baroni and Lenci, 2010), only few other languages possess large-scale syntax-based DSMs (Padó and Utt, 2012; Šnajder et al., 2013). This paper develops methods that take advantage of the resource gradient between English and other languages, exploiting the higher-quality resources of the former to induce resources for target languages among the latter, by translating the word-link-word co-occurrences that underlie syntax-based DSMs. This directly provides a crosslingual method to con- struct syntax-based DSMs for target languages with- out any target language data, requiring only an En- glish syntax-based DSM and a translation lexicon. Such lexicons are available for many language pairs, and we outline a method to reduce ambiguity inherent in such dictionaries. We describe a set of multilin- gual methods that can combine corpus evidence from …

Content truncated to 500 words