Effective Batching for Recurrent Neural Network Grammars

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 4340–4352 August 1–6, 2021. ©2021 Association for Computational Linguistics 4340 Effective Batching for Recurrent Neural Network Grammars Hiroshi Noji Artificial Intelligence Research Center AIST hiroshi.noji@aist.go.jp Yohei Oseki Graduate School of Arts and Sciences University of Tokyo oseki@g.ecc.u-tokyo.ac.jp Abstract As a language model that integrates traditional symbolic operations and flexible neural repre- sentations, recurrent neural network grammars (RNNGs) have attracted great attention from both scientific and engineering perspectives. However, RNNGs are known to be harder to scale due to the difficulty of batched training. In this paper, we propose effective batching for RNNGs, where every operation is computed in parallel with tensors across multiple sentences. Our PyTorch implementation effectively em- ploys a GPU and achieves x6 speedup com- pared to the existing C++ DyNet implemen- tation with model-independent auto-batching. Moreover, our batched RNNG also acceler- ates inference and achieves x20-150 speedup for beam search depending on beam sizes. Finally, we evaluate syntactic generalization performance of the scaled RNNG against the LSTM baseline, based on the large training data of 100M tokens from English Wikipedia and the broad-coverage targeted syntactic eval- uation benchmark.1 1 Introduction Neural language models have an excellent word prediction ability, which motivates researchers to develop several analysis methods for fine-grained evaluation, aiming at understanding which linguis- tic abilities the models have acquired during train- ing (Linzen et al., 2016; Wilcox et al., 2018; Mar- vin and Linzen, 2018; Warstadt et al., 2020). So far, many efforts have been made on the evalua- tion of syntactic performance of models, including the abilities to resolve distant subject-verb num- ber agreement in English. Since neural language models are the foundation of contemporary NLP systems, building a language model having robust sentence processing abilities like humans is an 1Our RNNG implementation is available at https://github.com/aistairc/rnng-pytorch/. important goal, especially toward a system with human-like syntactic generalization abilities, not relying on the data-specific superficial cues found in the training data (McCoy et al., 2019; Linzen, 2020). Past work has revealed that while sequential and unstructured models, such as LSTM and Trans- former language models (Hochreiter and Schmid- huber, 1997; Vaswani et al., 2017), can induce sev- eral interesting syntactic behaviors, there is also a notable advantage in explicitly modeling syntax with specific architectures (Kuncoro et al., 2018; Wilcox et al., 2019; Hu et al., 2020). The repre- sentative of such models is the recurrent neural network grammars (RNNGs; Dyer et al., 2016), the top-down, left-to-right generative models of a parse tree and sentence. While these results may suggest that RNNGs are a better modeling choice for language, unfor- tunately, they have a practical drawback in terms of scalability, due to their structure-sensitive com- putation mechanism (Kuncoro et al., 2019). Since the computational graphs of RNNGs depend on the tree structures of the sentences, training cannot be mini-batched easily. This is in contrast to LSTMs and Transformers, for which token-wise operations can be batched across sentences, allowing efficient computation on GPUs, which is the key …

Content truncated to 500 words