Neural Sequence Segmentation as Determining the Leftmost Segments

Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1476–1486 June 6–11, 2021. ©2021 Association for Computational Linguistics 1476 Neural Sequence Segmentation as Determining the Leftmost Segments Yangming Li1, Lemao Liu1, Kaisheng Yao2 1Tencent AI Lab 2Ant Group {newmanli,redmondliu}@tencent.com kaisheng.yao@antgroup.com Abstract Prior methods to text segmentation are mostly at token level. Despite the adequacy, this na- ture limits their full potential to capture the long-term dependencies among segments. In this work, we propose a novel framework that incrementally segments natural language sen- tences at segment level. For every step in seg- mentation, it recognizes the leftmost segment of the remaining sequence. Implementations involve LSTM-minus technique to construct the phrase representations and recurrent neu- ral networks (RNN) to model the iterations of determining the leftmost segments. We have conducted extensive experiments on syntactic chunking and Chinese part-of-speech (POS) tagging across 3 datasets, demonstrating that our methods have significantly outperformed previous all baselines and achieved new state- of-the-art results. Moreover, qualitative anal- ysis and the study on segmenting long-length sentences verify its effectiveness in modeling long-term dependencies. 1 Introduction Sequence segmentation, as an important task in natural language understanding (NLU), partitions a sentence into multiple segments. The first two rows of Table 1 show a case from a syntactic chunking dataset. The input sentence is a sequence of to- kens and the output segments are multiple labeled phrases. These segments are nonoverlapping and fully cover the input sentence. In previous works, there are two dominant ap- proaches to sequence segmentation. The most com- mon is to regard it as a sequence labeling prob- lem with resorting to IOB tagging scheme (Huang et al., 2015; Akbik et al., 2018; Liu et al., 2019). This method is simple yet very effective, provid- ing tons of state-of-the-art performances. For ex- ample, Huang et al. (2015) present Bidirectional LSTM-CRF for named entity recognition (NER) and POS tagging, which adopts BiLSTM (Hochre- Sentence Tangible capital will be about $ 115 million . Segments (Tangible capital, NP), (will be, VP), (about $ 115 million, NP), (., O) IOB Tags B-NP I-NP B-VP I-VP B-NP I-NP I-NP I-NP O SHIFT SHIFT REDUCE-NP Transition SHIFT SHIFT REDUCE-VP Actions SHIFT SHIFT SHIFT SHIFT REDUCE-NP OUT Table 1: The first two rows show an example extracted from CoNLL-2000 dataset (Sang and Buchholz, 2000). The last two rows are two types of token-level labels commonly used to represent the segments. iter and Schmidhuber, 1997) to read the input sen- tence and CRF (Lafferty et al., 2001) to decode the label sequence. An alternative method employs a transition-based system to incrementally segment and label an input token sequence (Zhang et al., 2016, 2018). For instance, Zhang et al. (2016) present a transition-based model for Chinese word segmentation that exploits not only character em- bedding but also token embedding. This type of method enjoys a number of attractive properties, including theoretically lower time complexity and capturing non-local features. The above two approaches are essentially …

Content truncated to 500 words