Unsupervised Mining of Analogical Frames by Constraint Satisfaction

Unsupervised Mining of Analogical Frames by Constraint Satisfaction Lance De Vine Shlomo Geva Peter Bruza Queensland University of Technology Brisbane, Australia {l.devine, s.geva, p.bruza}@qut.edu.au Abstract It has been demonstrated that vector-based representations of words trained on large text corpora encode linguistic regularities that may be exploited via the use of vector space arith- metic. This capability has been extensively explored and is generally measured via tasks which involve the automated completion of linguistic proportional analogies. The ques- tion remains, however, as to what extent it is possible to induce relations from word embed- dings in a principled and systematic way, with- out the provision of exemplars or seed terms. In this paper we propose an extensible and effi- cient framework for inducing relations via the use of constraint satisfaction. The method is efficient, unsupervised and can be customized in various ways. We provide both quantitative and qualitative analysis of the results. 1 Introduction The use and study of analogical inference and structure has a long history in linguistics, logic, cognitive psychology, scientific reasoning and ed- ucation (Bartha, 2016), amongst others. The use of analogy has played an especially important role in the study of language, language change, and language acquisition and learning (Kiparsky, 1992). It has been a part of the study of phonology, morphology, orthography and syntactic grammar (Skousen, 1989), as well as the development of applications such as machine translation and para- phasing (Lepage and Denoual, 2005). Recent progress with the construction of vec- tor space representations of words based on their distributional profiles has revealed that analogi- cal structure can be discovered and operationalised via the use of vector space algebra (Mikolov et al., 2013b). There remain many questions regarding the extent to which word vectors encode analog- ical structure and also the extent to which this structure can be uncovered. For example, we are not aware of any proposal or system that is fo- cussed on the unsupervised and systematic discov- ery of analogies from word vectors that does not make use of exemplar relations, existing linguistic resources or seed terms. The automatic identifica- tion of linguistic analogies, however, offers many potential benefits for a diverse range of research and applications, including language learning and computational creativity. Computational models of analogy have been studied since at least the 1960’s (Hall, 1989; French, 2002) and have addressed tasks relating to both proportional and structural analogy. Many computational systems are built as part of inves- tigations into how humans might perform analog- ical inference (Gentner and Forbus, 2011). Most make use of Structure Mapping Theory (SMT) (Gentner, 1983) or a variation thereof which maps one relational system to another, generally using a symbolic representation. Other systems use vector space representations constructed from corpora of natural language text (Turney, 2013). The analo- gies that are computed using word embeddings have primarily been proportional analogies and are closely associated with the prediction of relations between words. For example, a valid semantic proportional analogy is “cat is to feline as dog …

Content truncated to 500 words