Cross-Register Projection for Headline Part of Speech Tagging

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6475–6490 November 7–11, 2021. c⃝2021 Association for Computational Linguistics 6475 Cross-Register Projection for Headline Part of Speech Tagging Adrian Benton, Hanyang Li, Igor Malioutov Bloomberg 731 Lexington Ave New York, NY 10022 USA {abenton10,hli762,imalioutov}@bloomberg.net Abstract Part of speech (POS) tagging is a familiar NLP task. State of the art taggers routinely achieve token-level accuracies of over 97% on news body text, evidence that the problem is well- understood. However, the register of English news headlines, “headlinese”, is very different from the register of long-form text, causing POS tagging models to underperform on head- lines. In this work, we automatically annotate news headlines with POS tags by projecting predicted tags from corresponding sentences in news bodies. We train a multi-domain POS tagger on both long-form and headline text and show that joint training on both registers improves over training on just one or na¨ıvely concatenating training sets. We evaluate on a newly-annotated corpus of over 5,248 En- glish news headlines from the Google sentence compression corpus, and show that our model yields a 23% relative error reduction per to- ken and 19% per headline. In addition, we demonstrate that better headline POS tags can improve the performance of a syntax-based open information extraction system. We make POSH, the POS-tagged Headline corpus, avail- able to encourage research in improved NLP models for news headlines. 1 Introduction News headlines were identified as a unique reg- ister of written language at least as far back as Straumann (1935), and the term headlinese was coined specifically to refer to the unnatural register used in headlines. Hallmarks of headlinese include frequent omission of articles and auxiliary verbs, stand-alone nominals and adverbials, and infinitival forms of the main verb when referring to the future tense (M˚ardh, 1980). In spite of the clear differ- ences in syntax between news headlines and bod- ies, the NLP community has invested little effort in building headline-specific models for predicting syntactic annotations such as part of speech (POS) tags and dependency parse trees. This limits the use of syntax-based models on headlines. For example, the Open Domain Information Extraction (Open IE) system, PredPatt (White et al., 2016), cannot be expected to perform well on headlines, if existing models cannot accurately tag headlines with POS tags and dependency relations. More broadly, head- line processing provides an important signal for downstream applications such as summarization (Bambrick et al., 2020), sentiment/stance classifi- cation (Strapparava and Mihalcea, 2007; Ferreira and Vlachos, 2016), semantic clustering (Wities et al., 2017; Laban et al., 2021), and information re- trieval (Lee et al., 2010), among other applications. In this work we take a first step toward develop- ing strong NLP models for headlines by focusing on improving POS taggers. We propose a projec- tion technique (inspired by work on cross-lingual projection (Yarowsky et al., 2001)) to generate sil- ver training data for each headline based on POS tags from the lead sentence of the news body. To …

Content truncated to 500 words