Context Encoder for Analogies on Strings

Context Encoder for Analogies on Strings Tianjing Zhao Graduate School of Information, Production and Systems, Waseda University bilibilimisaka@akane.waseda.jp Yves Lepage Graduate School of Information, Production and Systems, Waseda University yves.lepage@waseda.jp Abstract We propose a model based on context en- coder to solve formal analogies on strings like aaabbbccc : aaaabbbbcccc :: abc : x ⇒ x = aabbcc or ubid : tubid :: ofjid : x ⇒ x = tofjid. As a context encoder model, it consists of a generator and a dis- criminator. The generator attempts at generat- ing the result of an analogical equation, while the discriminator attempts at discriminating solutions coming out of the generator against the real solution of the analogical equation. We conduct experiments on publicly available data sets to compare the performance of our model with a previously published method de- signed for the same task. Our results show slight increases in accuracy, in comparison to a fully connected neural network architecture. 1 Introduction To design machines or programs capable of solv- ing analogical equations is still a challenge for arti- ficial intelligence. There are several levels on which analogies can be solved: • world knowledge level: England : English :: Lichtenstein : x ⇒ x = German; • semantic level: man : woman :: king : x ⇒ x = queen; • grammatical level: I play : she plays :: I am : x ⇒ x = she is; • morphological level: cat : cats :: child : x ⇒ x = children. As for semantic analogies, the work in (Turney and Littman, 2005; Turney and Pantel, 2010) has shown that a computer program can reach average human performance in the task of solving SAT ques- tions. Nowadays, word embeddings are used to ad- dress this problem (e.g. (Drozd et al., 2016)). The example analogies given above are all irregu- lar in form. However, a number of analogies which hold at these same levels may exhibit regularities in the word forms themselves. • England : English :: Poland : x ⇒ x = Polish; • man : woman :: policeman : x ⇒ x = policewoman; • I play : she plays :: I accept : x ⇒ x = she accepts; • cat : cats :: dog : x ⇒ x = dogs. Such regularities account for a good part of lan- guage productivity, i.e., the creation of new word forms from old ones. The regularities encoun- tered in such new word forms can be described in terms of prefixing, suffixing, parallel infixing and circumfixing. This can be reduced to funda- mental edit operations found in defining edit dis- tances (Levenshtein, 1966): insertion, deletions and substitutions. With such a formalisation, the prob- lem becomes purely formal, and can be illustrated with examples not linked to any language, like: aaabbbccc : aaaabbbbcccc :: abc : x ⇒ x = aabbcc or lbmc : lvmbjc :: nvtkje : x ⇒ x = nvtbjkje. PACLIC 32 832 32nd Pacific Asia Conference on Language, Information and …

Content truncated to 500 words