Comparing Convolutional Neural Networks to Traditional Models for Slot Filling

Proceedings of NAACL-HLT 2016, pages 828–838, San Diego, California, June 12-17, 2016. c⃝2016 Association for Computational Linguistics Comparing Convolutional Neural Networks to Traditional Models for Slot Filling Heike Adel and Benjamin Roth and Hinrich Sch¨utze Center for Information and Language Processing (CIS) LMU Munich Oettingenstr. 67, 80538 Munich, Germany heike@cis.lmu.de Abstract We address relation classification in the con- text of slot filling, the task of finding and eval- uating fillers like “Steve Jobs” for the slot X in “X founded Apple”. We propose a convo- lutional neural network which splits the input sentence into three parts according to the re- lation arguments and compare it to state-of- the-art and traditional approaches of relation classification. Finally, we combine different methods and show that the combination is bet- ter than individual approaches. We also ana- lyze the effect of genre differences on perfor- mance. 1 Introduction Structured knowledge about the world is useful for many natural language processing (NLP) tasks, such as disambiguation, question answering or semantic search. However, the extraction of structured infor- mation from natural language text is challenging be- cause one relation can be expressed in many differ- ent ways. The TAC Slot Filling (SF) Shared Task defines slot filling as extracting fillers for a set of predefined relations (“slots”) from a large corpus of text data. Exemplary relations are the city of birth of a person or the employees or founders of a com- pany. Participants are provided with an evaluation corpus and a query file consisting of pairs of enti- ties and slots. For each entity slot pair (e.g. “Ap- ple” and “founded by”), the systems have to return the second argument (“filler”) of the relation (e.g. “Steve Jobs”) as well as a supporting sentence from the evaluation corpus. The key challenge in slot filling is relation classification: given a sentence s of the evaluation corpus containing the name of a queried entity (e.g., “Apple”) and a filler candidate (e.g., “Steve Jobs”), we need to decide whether s ex- presses the relation (“founded by”, in this case). We will refer to the mentions of the two arguments of the relation as name and filler. Performance on re- lation classification is crucial for slot filling since its effectiveness directly depends on it. In this paper, we investigate three complementary approaches to relation classification. The first approach is pattern matching, a leading approach in the TAC evaluations. Fillers are vali- dated based on patterns. In this work, we consider patterns learned with distant supervision and pat- terns extracted from Universal Schema relations. The second approach is support vector machines. We evaluate two different feature sets: a bag-of- word feature set (BOW) and more sophisticated skip n-gram features. Our third approach is a convolutional neural net- work (CNN). CNNs have been applied to NLP tasks like sentiment analysis, part-of-speech tagging and semantic role labeling. They can recognize phrase patterns independent of their position in the sen- tence. Furthermore, they make use of word embed- dings that directly reflect word similarity …

Content truncated to 500 words