Modeling Latent-Dynamic in Shallow Parsing: A Latent Conditional Model with Improved Inference

Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), pages 841–848 Manchester, August 2008 Modeling Latent-Dynamic in Shallow Parsing: A Latent Conditional Model with Improved Inference Xu Sun† Louis-Philippe Morency‡ Daisuke Okanohara† Jun’ichi Tsujii†§ †Department of Computer Science, The University of Tokyo, Hongo 7-3-1, Tokyo, Japan ‡USC Institute for Creative Technologies, 13274 Fiji Way, Marina del Rey, USA §School of Computer Science, The University of Manchester, 131 Princess St, Manchester, UK †{sunxu, hillbig, tsujii}@is.s.u-tokyo.ac.jp ‡morency@ict.usc.edu Abstract Shallow parsing is one of many NLP tasks that can be reduced to a sequence la- beling problem. In this paper we show that the latent-dynamics (i.e., hidden sub- structure of shallow phrases) constitutes a problem in shallow parsing, and we show that modeling this intermediate structure is useful. By analyzing the automatically learned hidden states, we show how the latent conditional model explicitly learn latent-dynamics. We propose in this paper the Best Label Path (BLP) inference algo- rithm, which is able to produce the most probable label sequence on latent condi- tional models. It outperforms two existing inference algorithms. With the BLP infer- ence, the LDCRF model significantly out- performs CRF models on word features, and achieves comparable performance of the most successful shallow parsers on the CoNLL data when further using part-of- speech features. 1 Introduction Shallow parsing identifies the non-recursive cores of various phrase types in text. The paradigmatic shallow parsing problem is noun phrase chunking, in which the non-recursive cores of noun phrases, called base NPs, are identified. As the represen- tative problem in shallow parsing, noun phrase chunking has received much attention, with the de- velopment of standard evaluation datasets and with c⃝ 2008. Licensed under the Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported li- cense (http://creativecommons.org/licenses/by-nc-sa/3.0/). Some rights reserved. extensive comparisons among methods (McDon- ald 2005; Sha & Pereira 2003; Kudo & Matsumoto 2001). Syntactic contexts often have a complex under- lying structure. Chunk labels are usually far too general to fully encapsulate the syntactic behavior of word sequences. In practice, and given the lim- ited data, the relationship between specific words and their syntactic contexts may be best modeled at a level finer than chunk tags but coarser than lexical identities. For example, in the noun phrase (NP) chunking task, suppose that there are two lex- ical sequences, “He is her –” and “He gave her – ”. The observed sequences, “He is her” and “He gave her”, would both be conventionally la- beled by ‘BOB’, where B signifies the ‘beginning NP’, and O the ‘outside NP’. However, this label- ing may be too general to encapsulate their respec- tive syntactic dynamics. In actuality, they have dif- ferent latent-structures, crucial in labeling the next word. For “He is her –”, the NP started by ‘her’ is still incomplete, so the label for – is likely to be I, which conveys the continuation of the phrase, e.g., “[He] is [her brother]”. In contrast, for “He gave her –”, the phrase started by ‘her’ is normally self- complete, …

Content truncated to 500 words