Active Learning Strategies for Support Vector Machines, Application to Temporal Relation Classification

Proceedings of the 5th International Joint Conference on Natural Language Processing, pages 56–64, Chiang Mai, Thailand, November 8 – 13, 2011. c⃝2011 AFNLP Active Learning Strategies for Support Vector Machines, Application to Temporal Relation Classification Seyed Abolghasem Mirroshandel⋆,† Gholamreza Ghassem-Sani⋆ Alexis Nasr† ⋆Computer Engineering Department, Sharif university of Technology, Tehran, Iran †Laboratoire d’Informatique Fondamentale de Marseille- CNRS - UMR 6166 Universit´e Aix-Marseille, Marseille, France (ghasem.mirroshandel@lif.univ-mrs.fr, sani@sharif.edu, alexis.nasr@lif.univ-mrs.fr) Abstract Temporal relations between events is a valu- able source of information which can be used in a large number of natural language pro- cessing applications such as question answer- ing, summarization, and information extrac- tion. Supervised temporal relation classifica- tion requires large corpora which are difficult, time consuming, and expensive to produce. Active learning strategies are well-suited to reduce this effort by efficiently selecting the most informative samples for labeling. This paper presents novel active learning strategies based on support vector machines (SVM) for temporal relation classification. A large num- ber of empirical comparisons of different ac- tive learning algorithms and various kernel functions in SVM shows that proposed active learning strategies are effective for the given task. 1 Introduction The identification of temporal relations between events, in texts, is a valuable information for many natural language processing (NLP) tasks, such as summarization, question answering, and informa- tion extraction. In question answering, one expects the system to answer questions such as “when an event occurred”, or “what is the chronological or- der of some desired events”. In text summarization, especially in the multi-document type, knowing the order of events is important for correctly merging related information. Most existing algorithms for temporal relation learning are supervised, they rely on manual annota- tions of corpora. Producing such annotated corpora has shown to be a time consuming, hard, and expen- sive task (Mani et al., 2006). In this paper we ex- plore active learning techniques as a way to control and speed up the annotation process. In the active learning framework, the learner has control over choosing the instances that will consti- tute the training set. A typical active learning algo- rithm begins with a small number of annotated data, and selects one or more informative instances from a large set of unlabeled instances, named the pool. The chosen instance(s) are then labeled and added to other annotated data, and the model is updated with this new information. These steps are repeated until at least one termination condition is satisfied. While there have been numerous applications of active learning to NLP researches (Settles and Craven, 2008; Xu et al., 2007), it has not been ap- plied, to our knowledge, to temporal relation classi- fication. This paper presents a novel active learning strat- egy for SVM-based classification algorithm. The proposed algorithm considers three measures: un- certainty, representativeness, and diversity to select the instances that will be annotated. The method we propose is generic, it could be applied to any SVM based classification problem. Temporal rela- tion classification has been selected, in this paper, for illustration purpose. …

Content truncated to 500 words