Neural semi-Markov CRF for Monolingual Word Alignment

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 6815–6828 August 1–6, 2021. ©2021 Association for Computational Linguistics 6815 Neural semi-Markov CRF for Monolingual Word Alignment Wuwei Lan⋆1, Chao Jiang⋆2, Wei Xu2 1 Department of Computer Science and Engineering, Ohio State University 2 School of Interactive Computing, Georgia Institute of Technology lan.105@osu.edu {chao.jiang, wei.xu}@cc.gatech.edu Abstract Monolingual word alignment is important for studying fine-grained editing operations (i.e., deletion, addition, and substitution) in text- to-text generation tasks, such as paraphrase generation, text simplification, neutralizing bi- ased language, etc. In this paper, we present a novel neural semi-Markov CRF alignment model, which unifies word and phrase align- ments through variable-length spans. We also create a new benchmark with human annota- tions that cover four different text genres to evaluate monolingual word alignment models in more realistic settings. Experimental results show that our proposed model outperforms all previous approaches for monolingual word alignment as well as a competitive QA-based baseline, which was previously only applied to bilingual data. Our model demonstrates good generalizability to three out-of-domain datasets and shows great utility in two down- stream applications: automatic text simplifi- cation and sentence pair classification tasks.1 1 Introduction Monolingual word alignment aims to align words or phrases with similar meaning in two sentences that are written in the same language. It is useful for improving the interpretability in nat- ural language understanding tasks, including se- mantic textual similarity (Li and Srikumar, 2016) and question answering (Yao, 2014). Monolin- gual word alignment can also support the anal- ysis of human editing operations (Figure 1) and improve model performance for text-to-text gen- eration tasks, such as text simplification (Mad- dela et al., 2021) and neutralizing biased language (Pryzant et al., 2020). It has also been shown to be helpful for data augmentation and label projection 1Our code and data will be available at: https:// github.com/chaojiang06/neural-Jacana ⋆Authors contributed equally. With Canadian collaborators, Lloyd went on to conduct laboratory simulations of his model. Lloyd performed successful laboratory experiments of his model. Deletion Deletion Substitution Insertion Figure 1: An example that illustrates monolingual word alignment (shown as arrows) can support analysis of human editing process and training of text generation models (§6.1), such as for simplifying complex sen- tences for children to read. (Culkin et al., 2021) when combined with para- phrase generation. One major challenge for automatic alignment is the need to handle not only alignments between words and linguistic phrases (e.g., a dozen ↔ more than 10), but also non-linguistic phrases that are semantically related given the context (e.g., tensions ↔ relations being strained in Figure 3). In this paper, we present a novel neural semi- Markov CRF alignment model, which unifies both word and phrase alignments though variable- length spans, calculates span-based semantic sim- ilarities, and takes alignment label transitions into consideration. We also create a new manually annotated benchmark, Multi-Genre Monolingual Word Alignment (MultiMWA), which consists of four datasets across different text …

Content truncated to 500 words