Identifying semantic relations in a specialized corpus through distributional analysis of a cooccurrence tensor

Identifying semantic relations in a specialized corpus through distributional analysis of a cooccurrence tensor Gabriel Bernier-Colborne OLST (Universit´e de Montr´eal) C.P. 6128, succ. Centre-Ville Montr´eal (Qu´ebec) Canada H3C 3J7 gabriel.bernier-colborne@umontreal.ca Abstract We describe a method of encoding cooc- currence information in a three-way tensor from which HAL-style word space mod- els can be derived. We use these models to identify semantic relations in a specialized corpus. Results suggest that the tensor- based methods we propose are more ro- bust than the basic HAL model in some respects. 1 Introduction Word space models such as LSA (Landauer and Dumais, 1997) and HAL (Lund et al., 1995) have been shown to identify semantic relations from corpus data quite effectively. However, the per- formance of such models depends on the parame- ters used to construct the word space. In the case of HAL, parameters such as the size of the con- text window can have a significant impact on the ability of the model to identify semantic relations and on the types of relations (e.g. paradigmatic or syntagmatic) captured. In this paper, we describe a method of encoding cooccurrence information which employs a three- way tensor instead of a matrix. Because the ten- sor explicitly encodes the distance between a tar- get word and the context words that co-occur with it, it allows us to extract matrices corresponding to HAL models with different context windows with- out repeatedly processing the whole corpus, but it also allows us to experiment with different kinds of word spaces. We describe one method whereby features are selected in different slices of the ten- sor corresponding to different distances between the target and context words, and another which uses SVD for dimensionality reduction. Models This work is licensed under a Creative Commons Attribution 4.0 International Licence. Page numbers and proceedings footer are added by the organisers. Licence details: http: //creativecommons.org/licenses/by/4.0/ are evaluated and compared on reference data ex- tracted from a specialized dictionary of the envi- ronment domain, as our target application is the identification of lexico-semantic relations in spe- cialized corpora. Preliminary results suggest the tensor-based methods are more robust than the ba- sic HAL model in some respects. 2 Related Work The tensor encoding method we describe is based on the Hyperspace Analogue to Language, or HAL, model (Lund et al., 1995; Lund and Burgess, 1996), which has been shown to be par- ticularly effective at modeling paradigmatic rela- tions such as synonymy. In the HAL model, word order is taken into account insofar as the word vec- tors it produces contain information about both the cooccurrents that precede a word and those that follow it. In recent years, there have been several proposals that aim to add word order information to models that rely mainly on word context infor- mation (Jones and Mewhort, 2007; Sahlgren et al., 2008), including models based on multi-way ten- sors. Symonds et al. (2011) proposed an efficient tensor encoding method which builds on unstruc- tured word space models (i.e. models …

Content truncated to 500 words