Phrase Chunking Using Entropy Guided Transformation Learning

Proceedings of ACL-08: HLT, pages 647–655, Columbus, Ohio, USA, June 2008. c⃝2008 Association for Computational Linguistics Phrase Chunking using Entropy Guided Transformation Learning Ruy L. Milidi´u Departamento de Inform´atica PUC-Rio Rio de Janeiro, Brazil milidiu@inf.puc-rio.br C´ıcero Nogueira dos Santos Departamento de Inform´atica PUC-Rio nogueira@inf.puc-rio.br Julio C. Duarte Centro Tecnol´ogico do Ex´ercito Rio de Janeiro, Brazil jduarte@ctex.eb.br Abstract Entropy Guided Transformation Learning (ETL) is a new machine learning strategy that combines the advantages of decision trees (DT) and Transformation Based Learn- ing (TBL). In this work, we apply the ETL framework to four phrase chunking tasks: Por- tuguese noun phrase chunking, English base noun phrase chunking, English text chunking and Hindi text chunking. In all four tasks, ETL shows better results than Decision Trees and also than TBL with hand-crafted tem- plates. ETL provides a new training strat- egy that accelerates transformation learning. For the English text chunking task this corre- sponds to a factor of five speedup. For Por- tuguese noun phrase chunking, ETL shows the best reported results for the task. For the other three linguistic tasks, ETL shows state-of-the- art competitive results and maintains the ad- vantages of using a rule based system. 1 Introduction Phrase Chunking is a Natural Language Processing (NLP) task that consists in dividing a text into syn- tactically correlated parts of words. Theses phrases are non-overlapping, i.e., a word can only be a mem- ber of one chunk (Sang and Buchholz, 2000). It pro- vides a key feature that helps on more elaborated NLP tasks such as parsing and information extrac- tion. Since the last decade, many high-performance chunking systems were proposed, such as, SVM- based (Kudo and Matsumoto, 2001; Wu et al., 2006), Winnow (Zhang et al., 2002), voted- perceptrons (Carreras and M`arquez, 2003), Transformation-Based Learning (TBL) (Ramshaw and Marcus, 1999; Megyesi, 2002) and Hidden Markov Model (HMM) (Molina and Pla, 2002), Memory-based (Sang, 2002). State-of-the-art systems for English base noun phrase chunking and text chunking are based in statistical techniques (Kudo and Matsumoto, 2001; Wu et al., 2006; Zhang et al., 2002). TBL is one of the most accurate rule-based tech- niques for phrase chunking tasks (Ramshaw and Marcus, 1999; Ngai and Florian, 2001; Megyesi, 2002). On the other hand, TBL rules must follow patterns, called templates, that are meant to cap- ture the relevant feature combinations. The process of generating good templates is highly expensive. It strongly depends on the problem expert skills to build them. Even when a template set is available for a given task, it may not be effective when we change from a language to another (dos Santos and Oliveira, 2005). In this work, we apply Entropy Guided Transfor- mation Learning (ETL) for phrase chunking. ETL is a new machine learning strategy that combines the advantages of Decision Trees (DT) and TBL (dos Santos and Milidi´u, 2007a). The ETL key idea is to use decision tree induction to obtain feature com- binations (templates) and then use the TBL algo- rithm to generate transformation rules. ETL pro- …

Content truncated to 500 words