Feature-Rich Two-Stage Logistic Regression for Monolingual Alignment

Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 949–959, Lisbon, Portugal, 17-21 September 2015. c⃝2015 Association for Computational Linguistics. Feature-Rich Two-Stage Logistic Regression for Monolingual Alignment Md Arafat Sultan†, Steven Bethard‡ and Tamara Sumner† †Institute of Cognitive Science and Department of Computer Science University of Colorado Boulder ‡Department of Computer and Information Sciences University of Alabama at Birmingham arafat.sultan@colorado.edu, bethard@cis.uab.edu, sumner@colorado.edu Abstract Monolingual alignment is the task of pair- ing semantically similar units from two pieces of text. We report a top-performing supervised aligner that operates on short text snippets. We employ a large feature set to (1) encode similarities among semantic units (words and named entities) in context, and (2) address cooperation and competi- tion for alignment among units in the same snippet. These features are deployed in a two-stage logistic regression framework for alignment. On two benchmark data sets, our aligner achieves F1 scores of 92.1% and 88.5%, with statistically significant er- ror reductions of 4.8% and 7.3% over the previous best aligner. It produces top re- sults in extrinsic evaluation as well. 1 Introduction Computer applications frequently require seman- tic comparison between short snippets of natural language text. Such comparisons are key to para- phrase detection (Das and Smith, 2009; Madnani et al., 2012), textual similarity identification (Agirre et al., 2015; Sultan et al., 2015) and recognition of textual entailment (Dagan and Glickman, 2004; Pad´o et al., 2015). And they underpin applications such as short answer grading (Mohler et al., 2011), question answering (Hixon et al., 2015), machine translation evaluation (Pad´o et al., 2009), and ma- chine reading (de Marneffe et al., 2007). A central problem underlying all text compari- son tasks is that of alignment: pairing related se- mantic units (i.e. words and phrases) across the two snippets (MacCartney et al., 2008; Thadani and McKeown, 2011; Thadani et al., 2012; Yao et al., 2013a; Yao et al., 2013b; Sultan et al., 2014a). Studies have shown that such tasks can benefit from an explicit alignment component (Hickl and Bens- ley, 2007; Sultan et al., 2014b; Sultan et al., 2015). However, alignment is still an open research prob- lem. We present a supervised monolingual aligner that produces top results in several intrinsic and extrinsic evaluation experiments. We pinpoint a set of key challenges for alignment and design a model with components targeted at each. Lexical and phrasal alignments can both be rep- resented as pairs of words – in the form of many- to-many mappings among the two phrases’ com- ponent words in the latter case. Thus without loss of generality, we formulate alignment as a binary classification task where given all word pairs across two sentences, the goal is to assign each a class la- bel in {aligned, not aligned}. However, this is not a straightforward classification scenario where each word pair can be treated independently – words in the same snippet can play both mutually cooper- ating and competing roles in complex ways. For example, semantically similar words in a snippet …

Content truncated to 500 words