A Document-Level Neural Machine Translation Model with Dynamic Caching Guided by Theme-Rheme Information

Proceedings of the 28th International Conference on Computational Linguistics, pages 426–436 Barcelona, Spain (Online), December 8-13, 2020 426 Generating Equation by Utilizing Operators : GEO Model Kyung Seo Ki* Donggeon Lee* Bugeun Kim Gahgene Gweon Department of Transdisciplinary Studies Seoul National University Seoul, Republic of Korea {kskee88,lsw5835,cd4209,ggweon}@snu.ac.kr Abstract Math word problem solving is an emerging research topic in Natural Language Processing. Re- cently, to address the math word problem solving task, researchers have applied the encoder- decoder architecture, which is mainly used in machine translation tasks. The state-of-the-art neural models use hand-crafted features and are based on generation methods. In this paper, we propose the GEO (Generation of Equations by utilizing Operators) model that does not use hand- crafted features and addresses two issues that are present in existing neural models: 1. missing domain-specific knowledge features and 2. losing encoder-level knowledge. To address missing domain-specific feature issue, we designed two auxiliary tasks: operation group difference pre- diction and implicit pair prediction. To address losing encoder-level knowledge issue, we added an Operation Feature Feed Forward (OP3F) layer. Experimental results showed that the GEO model outperformed existing state-of-the-art models on two datasets, 85.1% in MAWPS, and 62.5% in DRAW-1K, and reached comparable performance of 82.1% in ALG514 dataset. 1 Introduction Math word problem solving is an emerging research topic in Natural Language Processing. The problem solving task involves dealing with numbers or numeric context information, which is useful in under- standing language. Recently, to address the math word problem solving task, researchers applied neural models with the encoder-decoder architecture, which is mainly used in machine translation tasks. Such models generate equations as outputs using given natural language sentence sequences as inputs (Wang et al., 2019; Meng and Rumshisky, 2019). The state-of-the-art neural models use hand-crafted features and are based on generation methods. In the primary benchmark datasets, such hand-crafted feature- based models have recently reported performance levels of about 80%, while neural models remain at 60-70% (Wang et al., 2017). Due to the relatively low performance of neural models, models that utilize hand-crafted features still retain state-of-the-art (SOTA) performance in many datasets (Upadhyay et al., 2016a). However, defining and building hand-crafted features is costly. In this paper, we propose the GEO (Generation of Equations by utilizing Operators) model that does not use hand-crafted features and addresses two issues that are present in existing neural models: 1. missing domain-specific knowledge features and 2. losing encoder-level knowledge. The first issue of missing domain-specific knowledge feature occurs when neural models don’t make enough use of domain-specific information of the target domain. For our case of math word problem solving, domain-specific knowledge is mathematical information. Considering how humans extract and process mathematical information from a word problem when building an equation for the problem, people identify main mathematical elements, such as numbers and operations (Usiskin, 1988). Existing neural models so far assumes that such main mathematical elements would be automatically identified in the model training process. Yet, explicitly utilizing information about numbers and operations can be …

Content truncated to 500 words