Luminoso at SemEval-2018 Task 10: Distinguishing Attributes Using Text Corpora and Relational Knowledge

Luminoso at SemEval-2018 Task 10: Distinguishing Attributes Using Text Corpora and Relational Knowledge Robyn Speer Luminoso Technologies, Inc. 675 Massachusetts Avenue Cambridge, MA 02139 rspeer@luminoso.com Joanna Lowry-Duda Luminoso Technologies, Inc. 675 Massachusetts Avenue Cambridge, MA 02139 jlowry-duda@luminoso.com Abstract Luminoso participated in the SemEval 2018 task on “Capturing Discriminative Attributes” with a system based on ConceptNet, an open knowledge graph focused on general knowl- edge. In this paper, we describe how we trained a linear classifier on a small number of semantically-informed features to achieve an F1 score of 0.7368 on the task, close to the task’s high score of 0.75. 1 Introduction Word embeddings are most effective when they learn from both unstructured text and a graph of general knowledge (Speer and Lowry-Duda, 2017). ConceptNet 5 (Speer et al., 2017) is an open-data knowledge graph that is well suited for this purpose. It is accompanied by a pre-built word embedding model known as ConceptNet Number- batch1, which combines skip-gram embeddings learned from unstructured text with the relational knowledge in ConceptNet. A straightforward application of the Concept- Net Numberbatch embeddings took first place in SemEval 2017 task 2, on semantic word similarity. For SemEval 2018, we built a system with these embeddings as a major component for a slightly more complex task. The Capturing Discriminative Attributes task (Paperno et al., 2018) emphasizes the ability of a semantic model to recognize relevant differences between terms, not just their similarities. As the task description states, “If you can tell that ameri- cano is similar to capuccino and espresso but you can’t tell the difference between them, you don’t know what americano is.” The ConceptNet Numberbatch embeddings only measure the similarity of terms, and we hy- 1https://github.com/commonsense/ conceptnet-numberbatch pothesized that we would need to represent more specific relationships. For example, the input triple “frog, snail, legs” asks us to determine whether “legs” is an attribute that distinguishes “frog” from “snail”. The answer is yes, because a frog has legs while a snail does not. The has re- lationship is one example of a specific relationship that is represented in ConceptNet. To capture this kind of specific relationship, we built a model that infers relations between Con- ceptNet nodes, trained on the existing edges in ConceptNet and random negative examples. There are many models designed for this purpose; the one we decided on is based on Semantic Matching Energy (SME) (Bordes et al., 2014). Our features consisted of direct similarity over ConceptNet Numberbatch embeddings, the rela- tionships inferred over ConceptNet by SME, fea- tures that compose ConceptNet with other re- sources (WordNet and Wikipedia), and a purely corpus-based feature that looks up two-word phrases in the Google Books dataset. We combined these features based on Concept- Net with features extracted from a few other re- sources in a LinearSVC classifier, using liblinear (Fan et al., 2008) via scikit-learn (Pedregosa et al., 2011). The classifier used only 15 features, of which 12 ended up with non-zero weights, from the five sources described. We aimed to avoid …

Content truncated to 500 words