LiMiT: The Literal Motion in Text Dataset

Findings of the Association for Computational Linguistics: EMNLP 2020, pages 84–93 November 16 - 20, 2020. c⃝2020 Association for Computational Linguistics 84 Improving Text Understanding via Deep Syntax-Semantics Communication Hao Fei1, Yafeng Ren2 and Donghong Ji1∗ 1. Department of Key Laboratory of Aerospace Information Security and Trusted Computing, Ministry of Education, School of Cyber Science and Engineering, Wuhan University, China 2. Guangdong University of Foreign Studies, China {hao.fei,renyafeng,dhji}@whu.edu.cn Abstract Recent studies show that integrating syntactic tree models with sequential semantic models can bring improved task performance, while these methods mostly employ shallow inte- gration of syntax and semantics. In this pa- per, we propose a deep neural communica- tion model between syntax and semantics to improve the performance of text understand- ing. Local communication is performed be- tween syntactic tree encoder and sequential se- mantic encoder for mutual learning of informa- tion exchange. Global communication can fur- ther ensure comprehensive information propa- gation. Results on multiple syntax-dependent tasks show that our model outperforms strong baselines by a large margin. In-depth analysis indicates that our method is highly effective in composing sentence semantics. 1 Introduction Neural sequential models such as LSTM (Hochre- iter and Schmidhuber, 1997), GRU (Cho et al., 2014), Transformer (Vaswani et al., 2017), ELMo (Peters et al., 2018) and BERT (Devlin et al., 2019), have been extensively applied for encoding the semantics of texts in natural language processing (NLP) (Sundermeyer et al., 2012; Bahdanau et al., 2015; Dozat and Manning, 2017; Yuan et al., 2019). On the other hand, hierarchical tree models, such as TreeLSTM (Socher et al., 2013) and GCN (Kipf and Welling, 2017), have been introduced to en- rich sequence encoding with syntactic information, bringing further strengths in text modeling. Such external syntactic structure knowledge provides en- hanced features, which can facilitate a broad range of NLP tasks (Tai et al., 2015; Looks et al., 2017; Zhang and Zhang, 2019). Recent studies show that integrating syntactic tree models with sequential semantic models can ∗Corresponding author. B. James loves the atmosphere in the city A. James witnessed the tragedy in the city · Syntactic √ · Semantic × Similar: (a) The same syntactic structure but different semantics. B. Tom cooked his own past under threat A. They forced Tom to fake his history · Syntactic × · Semantic √ Similar: (b) The similar semantics but different syntactic structures. Figure 1: Comparisons of syntax and semantics in sen- tences. The same color indicates the same (similar) se- mantic objective. bring improved performance for syntax-dependent tasks (Shi et al., 2016; Havrylov et al., 2019), such as semantic role labeling (SRL) (Wang et al., 2019) and natural language inference (NLI) (Chen et al., 2017; Liu et al., 2018, 2019), etc. Intuitively, se- quential semantic models and syntactic tree models play different roles in text modeling. Sequential semantic models learn the representation via adja- cency neighborhood, while syntactic tree models encode texts through structural connections. Tak- ing the two sentence pairs from the NLI task in Fig- ure 1 as example, …

Content truncated to 500 words