Contrasting Syntagmatic and Paradigmatic Relations: Insights from Distributional Semantic Models

Proceedings of the Third Joint Conference on Lexical and Computational Semantics (*SEM 2014), pages 160–170, Dublin, Ireland, August 23-24 2014. Contrasting Syntagmatic and Paradigmatic Relations: Insights from Distributional Semantic Models Gabriella Lapesa3,1 1Universit¨at Osnabr¨uck Institut f¨ur Kognitionswissenschaft glapesa@uos.de Stefan Evert2 2FAU Erlangen-N¨urnberg Professur f¨ur Korpuslinguistik stefan.evert@fau.de Sabine Schulte im Walde3 3Universit¨at Stuttgart Institut f¨ur Maschinelle Sprachverarbeitung schulte@ims.uni-stuttgart.de Abstract This paper presents a large-scale evalua- tion of bag-of-words distributional models on two datasets from priming experiments involving syntagmatic and paradigmatic relations. We interpret the variation in performance achieved by different settings of the model parameters as an indication of which aspects of distributional patterns characterize these types of relations. Con- trary to what has been argued in the litera- ture (Rapp, 2002; Sahlgren, 2006) – that bag-of-words models based on second- order statistics mainly capture paradig- matic relations and that syntagmatic rela- tions need to be gathered from first-order models – we show that second-order mod- els perform well on both paradigmatic and syntagmatic relations if their parameters are properly tuned. In particular, our re- sults show that size of the context window and dimensionality reduction play a key role in differentiating DSM performance on paradigmatic vs. syntagmatic relations. 1 Introduction Distributional takes on the representation and ac- quisition of word meaning rely on the assump- tion that words with similar meaning tend to oc- cur in similar contexts: this assumption, known as distributional hypothesis, has been first proposed by Harris (1954). Distributional Semantic Mod- els (henceforth, DSMs) are computational mod- els that operationalize the distributional hypoth- esis; they produce semantic representations for words in the form of distributional vectors record- ing patterns of co-occurrence in large samples of language data (Sahlgren, 2006; Baroni and Lenci, 2010; Turney and Pantel, 2010). Comparison be- tween distributional vectors allows the identifica- tion of shared contexts as an empirical correlate of the semantic similarity between the target words. As noted in Sahlgren (2008), the notion of seman- tic similarity applied in distributional approaches to meaning is an easy target of criticism, as it is employed to capture a wide range of semantic re- lations, such as synonymy, antonymy, hypernymy, up to topical relatedness. The study presented in this paper contributes to the debate concerning the nature of the seman- tic representations built by DSMs, and it does so by comparing the performance of several DSMs in a classification task conducted on priming data and involving paradigmatic and syntagmatic rela- tions. Paradigmatic relations hold between words that occur in similar contexts; they are also called relations in absentia (Sahlgren, 2006) because paradigmatically related words do not co-occur. Examples of paradigmatic relations are synonyms (e.g., frigid–cold) and antonyms (e.g., cold–hot). Syntagmatic relations hold between words that co- occur (relations in praesentia) and therefore ex- hibit a similar distribution across contexts. Typi- cal examples of syntagmatic relations are phrasal associates (e.g., help–wanted) and syntactic collo- cations (e.g., dog–bark). Distributional modeling has already tackled the issue of paradigmatic and syntagmatic relations (Sahlgren, 2006; Rapp, 2002). Key contributions of …

Content truncated to 500 words