CA-EHN: Commonsense Analogy from E-HowNet

Proceedings of the 12th Conference on Language Resources and Evaluation (LREC 2020), pages 2984–2990 Marseille, 11–16 May 2020 c⃝ European Language Resources Association (ELRA), licensed under CC-BY-NC 2984 CA-EHN: Commonsense Analogy from E-HowNet Peng-Hsuan Li, Tsan-Yu Yang, Wei-Yun Ma Academia Sinica jacobvsdanniel@iis.sinica.edu.tw, s0920331239@gmail.com, ma@iis.sinica.edu.tw Abstract Embedding commonsense knowledge is crucial for end-to-end models to generalize inference beyond training corpora. However, existing word analogy datasets have tended to be handcrafted, involving permutations of hundreds of words with only dozens of pre-defined relations, mostly morphological relations and named entities. In this work, we model commonsense knowledge down to word-level analogical reasoning by leveraging E-HowNet, an ontology that annotates 88K Chinese words with their structured sense definitions and English translations. We present CA-EHN, the first commonsense word analogy dataset containing 90,505 analogies covering 5,656 words and 763 relations. Experiments show that CA-EHN stands out as a great indicator of how well word representations embed commonsense knowledge. The dataset is publicly available at https://github.com/ckiplab/CA-EHN. Keywords: Corpus, Lexicon, Lexical Database, Ontologies 1. Introduction Commonsense reasoning is fundamental for natural lan- guage agents to generalize inference beyond training cor- pora. Although the natural language inference (NLI) task (Bowman et al., 2015; Williams et al., 2018) has proved a good pre-training objective for sentence representations (Conneau et al., 2017), commonsense coverage is implicit and limited by the amount of annotated sentence pairs. Fur- thermore, most models are still end-to-end, relying heav- ily on word representations to provide background world knowledge. Therefore, it is desirable to model commonsense knowl- edge down to word-level analogical reasoning. In this sense, existing analogy benchmarks are lackluster. For Chi- nese analogy (CA), the simplified Chinese dataset CA8 (Li et al., 2018) and the traditional Chinese dataset CA-Google (Chen and Ma, 2018), translated from English (Mikolov et al., 2013), contain only a few dozen relations, most of which are either morphological, e.g., a shared prefix, or about named entities, e.g., capital-country. However, commonsense knowledge bases such as WordNet (Miller, 1995) and ConceptNet (Speer and Havasi, 2012) have long annotated relations in our lexicon. Among them, E-HowNet (Chen et al., 2005; Ma and Shih, 2018), ex- tended from HowNet (Dong and Dong, 2003), currently annotates 88K traditional Chinese words with their struc- tured definitions and English translations. In this paper, we propose an algorithm to extract accu- rate analogies from E-HowNet with refinements from lin- guists. We present CA-EHN, the first commonsense anal- ogy dataset containing 90,505 analogies covering 5,656 words and 763 relations. In the experiments, we show that it is useful to embed more commonsense knowledge and that CA-EHN tests this aspect of word embedding. 2. Related Work In this work, we use word sense definitions from the E- HowNet (Chen et al., 2005; Ma and Shih, 2018) ontology as well as further linguist refinements to construct our com- monsense word analogy corpus. Compared to the WordNet (Miller, 1995) gloss, E-HowNet has structured definitions for word senses, each of which can be parsed into a defini- tion graph. These graphs are fundamentally …

Content truncated to 500 words